2024-09-21T23:23:41.100 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T23:23:41.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T23:23:41.194 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914530' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgeet36R+jfq/IFcfuETTNF5qTP7zALK6Dmvx1qL28s51kYmxcjdQ01FrlhzLucpHynO1QWGZE3XTjMZfYb/D0= smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBINzdY72cKL+DRCr5QLPb/7wgaDGwPqF2x3KuBPR9yJ8MMoARZcFR0bR+Y+X+9jDoi+KSoMEPN7K+UXDFz4PCKs= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJhYYT2yW1ohXK2PwQ58q8HJ8+z2zNphvoxX5w4M5Ax0J96jzVqJXDXkZuLNbdCKYd6wRruhnFIM/fDIYdIGjhA= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - workunit: clients: client.0: - rbd/rbd-nbd.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T23:23:41.194 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-21T23:23:41.195 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-21T23:23:41.195 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T23:23:41.196 INFO:teuthology.task.internal:Checking packages... 2024-09-21T23:23:41.217 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-21T23:23:41.217 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T23:23:41.217 INFO:teuthology.packaging:ref: None 2024-09-21T23:23:41.217 INFO:teuthology.packaging:tag: None 2024-09-21T23:23:41.217 INFO:teuthology.packaging:branch: squid 2024-09-21T23:23:41.217 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:23:41.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-21T23:23:41.431 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:23:41.433 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T23:23:41.441 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T23:23:41.441 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T23:23:41.494 INFO:teuthology.task.internal:Saving configuration 2024-09-21T23:23:41.507 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T23:23:41.528 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T23:23:41.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 23:17:06.211246', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILb8cHKmGXhDqJLsHy4i4w5Ltkop30sK3+ZuKCOcRmdJ'} 2024-09-21T23:23:41.567 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 23:17:06.212927', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDfrYma9AlX3ArQ7WqayLTxP1iwwDRuChF+4S/k235xe'} 2024-09-21T23:23:41.583 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 23:17:06.214247', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILndpSlFg8aV/hl+Hazjbdd1UnsRww502JtKH/b/a8x1'} 2024-09-21T23:23:41.583 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T23:23:41.590 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-21T23:23:41.590 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-21T23:23:41.590 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['client.0'] 2024-09-21T23:23:41.591 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T23:23:41.691 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efcbdd47880>, signals=[15]) 2024-09-21T23:23:41.691 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T23:23:41.700 INFO:teuthology.task.internal:Opening connections... 2024-09-21T23:23:41.700 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:23:41.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:23:41.784 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:23:41.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:23:41.863 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-09-21T23:23:41.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:23:41.943 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T23:23:41.952 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-09-21T23:23:41.958 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-09-21T23:23:41.958 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Ubuntu" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="22.04" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_CODENAME=jammy 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:ID=ubuntu 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE=debian 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:23:42.007 INFO:teuthology.orchestra.run.smithi079.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:23:42.008 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-09-21T23:23:42.029 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-09-21T23:23:42.035 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-09-21T23:23:42.035 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Ubuntu" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="22.04" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_CODENAME=jammy 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:ID=ubuntu 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE=debian 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:23:42.084 INFO:teuthology.orchestra.run.smithi118.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:23:42.085 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-09-21T23:23:42.113 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-09-21T23:23:42.118 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-09-21T23:23:42.119 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-09-21T23:23:42.164 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="Ubuntu" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="22.04" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_CODENAME=jammy 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:ID=ubuntu 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE=debian 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:23:42.165 INFO:teuthology.orchestra.run.smithi165.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:23:42.165 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-09-21T23:23:42.196 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T23:23:42.212 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T23:23:42.243 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T23:23:42.243 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:23:42.246 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:23:42.248 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:23:42.253 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T23:23:42.277 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T23:23:42.277 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:23:42.292 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:23:42.294 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:23:42.307 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T23:23:42.382 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T23:23:42.397 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T23:23:42.397 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T23:23:42.397 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:23:42.397 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:23:42.398 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:23:42.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:23:42.403 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-21T23:23:42.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:23:42.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:23:42.404 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-21T23:23:42.404 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-21T23:23:42.451 INFO:teuthology.orchestra.run.smithi079.stdout:5.15.0-56-generic 2024-09-21T23:23:42.451 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.15.0-56-generic 2024-09-21T23:23:42.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get clean 2024-09-21T23:23:42.452 INFO:teuthology.orchestra.run.smithi118.stdout:5.15.0-56-generic 2024-09-21T23:23:42.452 INFO:teuthology.task.kernel:Running kernel on smithi118: 5.15.0-56-generic 2024-09-21T23:23:42.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-09-21T23:23:42.453 INFO:teuthology.orchestra.run.smithi165.stdout:5.15.0-56-generic 2024-09-21T23:23:42.454 INFO:teuthology.task.kernel:Running kernel on smithi165: 5.15.0-56-generic 2024-09-21T23:23:42.454 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get clean 2024-09-21T23:23:42.556 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-21T23:23:42.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-21T23:23:42.705 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:23:42.705 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:23:42.705 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:23:42.744 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-21T23:23:42.774 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:23:42.854 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:23:42.862 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:23:42.920 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:23:42.942 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:23:43.013 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:23:43.013 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:23:43.042 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:23:43.050 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:23:43.052 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:23:43.054 INFO:teuthology.orchestra.run.smithi165.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:23:43.054 INFO:teuthology.orchestra.run.smithi165.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:23:43.076 INFO:teuthology.orchestra.run.smithi165.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:23:43.077 INFO:teuthology.orchestra.run.smithi165.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:23:43.079 INFO:teuthology.orchestra.run.smithi165.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:23:43.080 INFO:teuthology.orchestra.run.smithi165.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:23:43.089 INFO:teuthology.orchestra.run.smithi165.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:23:43.093 INFO:teuthology.orchestra.run.smithi165.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:23:43.094 INFO:teuthology.orchestra.run.smithi165.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:23:43.105 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:23:43.113 INFO:teuthology.orchestra.run.smithi165.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:23:43.114 INFO:teuthology.orchestra.run.smithi165.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:23:43.114 INFO:teuthology.orchestra.run.smithi165.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:23:43.114 INFO:teuthology.orchestra.run.smithi165.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:23:43.114 INFO:teuthology.orchestra.run.smithi165.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:23:43.118 INFO:teuthology.orchestra.run.smithi165.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:23:43.120 INFO:teuthology.orchestra.run.smithi165.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:23:43.120 INFO:teuthology.orchestra.run.smithi165.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:23:43.121 INFO:teuthology.orchestra.run.smithi165.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:23:43.139 INFO:teuthology.orchestra.run.smithi165.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:23:43.142 INFO:teuthology.orchestra.run.smithi165.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:23:43.142 INFO:teuthology.orchestra.run.smithi165.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:23:43.146 INFO:teuthology.orchestra.run.smithi165.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:23:43.153 INFO:teuthology.orchestra.run.smithi165.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:23:43.155 INFO:teuthology.orchestra.run.smithi165.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:23:43.155 INFO:teuthology.orchestra.run.smithi165.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:23:43.155 INFO:teuthology.orchestra.run.smithi165.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:23:43.156 INFO:teuthology.orchestra.run.smithi165.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:23:43.178 INFO:teuthology.orchestra.run.smithi165.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:23:43.179 INFO:teuthology.orchestra.run.smithi165.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:23:43.179 INFO:teuthology.orchestra.run.smithi165.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:23:43.180 INFO:teuthology.orchestra.run.smithi165.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:23:43.180 INFO:teuthology.orchestra.run.smithi165.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:23:43.189 INFO:teuthology.orchestra.run.smithi165.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:23:43.189 INFO:teuthology.orchestra.run.smithi165.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:23:43.189 INFO:teuthology.orchestra.run.smithi165.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:23:43.206 INFO:teuthology.orchestra.run.smithi165.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:23:43.276 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:23:43.297 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:23:43.415 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:23:43.461 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:23:43.482 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:23:43.600 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:23:43.620 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:23:43.723 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:23:43.740 INFO:teuthology.orchestra.run.smithi079.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:23:43.749 INFO:teuthology.orchestra.run.smithi079.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:23:43.749 INFO:teuthology.orchestra.run.smithi079.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:23:43.751 INFO:teuthology.orchestra.run.smithi079.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:23:43.770 INFO:teuthology.orchestra.run.smithi079.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:23:43.784 INFO:teuthology.orchestra.run.smithi079.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:23:43.785 INFO:teuthology.orchestra.run.smithi079.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:23:43.786 INFO:teuthology.orchestra.run.smithi079.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:23:43.822 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:23:43.847 INFO:teuthology.orchestra.run.smithi079.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:23:43.876 INFO:teuthology.orchestra.run.smithi079.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:23:43.876 INFO:teuthology.orchestra.run.smithi079.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:23:43.876 INFO:teuthology.orchestra.run.smithi079.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:23:43.884 INFO:teuthology.orchestra.run.smithi079.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:23:43.884 INFO:teuthology.orchestra.run.smithi079.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:23:43.893 INFO:teuthology.orchestra.run.smithi079.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:23:43.903 INFO:teuthology.orchestra.run.smithi079.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:23:43.905 INFO:teuthology.orchestra.run.smithi079.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:23:43.905 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:23:43.908 INFO:teuthology.orchestra.run.smithi079.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:23:43.909 INFO:teuthology.orchestra.run.smithi079.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:23:43.909 INFO:teuthology.orchestra.run.smithi079.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:23:43.911 INFO:teuthology.orchestra.run.smithi079.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:23:43.921 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:23:43.924 INFO:teuthology.orchestra.run.smithi079.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:23:43.929 INFO:teuthology.orchestra.run.smithi079.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:23:43.930 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:23:43.931 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:23:43.938 INFO:teuthology.orchestra.run.smithi079.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:23:43.939 INFO:teuthology.orchestra.run.smithi079.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:23:43.940 INFO:teuthology.orchestra.run.smithi079.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:23:43.955 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:23:43.969 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:23:43.970 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:23:43.971 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:23:43.988 INFO:teuthology.orchestra.run.smithi079.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:23:44.012 INFO:teuthology.orchestra.run.smithi079.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:23:44.013 INFO:teuthology.orchestra.run.smithi079.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:23:44.014 INFO:teuthology.orchestra.run.smithi079.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:23:44.015 INFO:teuthology.orchestra.run.smithi079.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:23:44.015 INFO:teuthology.orchestra.run.smithi079.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:23:44.019 INFO:teuthology.orchestra.run.smithi079.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:23:44.020 INFO:teuthology.orchestra.run.smithi079.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:23:44.027 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:23:44.028 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:23:44.090 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:23:44.090 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:23:44.090 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:23:44.098 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:23:44.098 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:23:44.101 INFO:teuthology.orchestra.run.smithi079.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:23:44.101 INFO:teuthology.orchestra.run.smithi079.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:23:44.102 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:23:44.111 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:23:44.118 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:23:44.121 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:23:44.122 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:23:44.123 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:23:44.130 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:23:44.134 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:23:44.135 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:23:44.136 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:23:44.136 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:23:44.171 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:23:44.172 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:23:44.200 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:23:44.200 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:23:44.202 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:23:44.203 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:23:44.282 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:23:44.282 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:23:44.283 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:23:44.283 INFO:teuthology.orchestra.run.smithi118.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:23:44.284 INFO:teuthology.orchestra.run.smithi118.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:23:49.408 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 16.3 MB in 2s (7,499 kB/s) 2024-09-21T23:23:50.163 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 16.3 MB in 3s (5,904 kB/s) 2024-09-21T23:23:50.307 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 16.3 MB in 3s (5,873 kB/s) 2024-09-21T23:23:50.834 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:23:50.851 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:23:50.913 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:23:51.104 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-21T23:23:51.105 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-21T23:23:51.272 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:23:51.272 INFO:teuthology.orchestra.run.smithi165.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:23:51.272 INFO:teuthology.orchestra.run.smithi165.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:23:51.272 INFO:teuthology.orchestra.run.smithi165.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:23:51.272 INFO:teuthology.orchestra.run.smithi165.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:23:51.273 INFO:teuthology.orchestra.run.smithi165.stdout: python2.7-minimal 2024-09-21T23:23:51.273 INFO:teuthology.orchestra.run.smithi165.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:23:51.274 INFO:teuthology.orchestra.run.smithi165.stdout:The following additional packages will be installed: 2024-09-21T23:23:51.274 INFO:teuthology.orchestra.run.smithi165.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:51.274 INFO:teuthology.orchestra.run.smithi165.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:23:51.274 INFO:teuthology.orchestra.run.smithi165.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:51.276 INFO:teuthology.orchestra.run.smithi165.stdout:Suggested packages: 2024-09-21T23:23:51.276 INFO:teuthology.orchestra.run.smithi165.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:23:51.276 INFO:teuthology.orchestra.run.smithi165.stdout:Recommended packages: 2024-09-21T23:23:51.276 INFO:teuthology.orchestra.run.smithi165.stdout: thermald 2024-09-21T23:23:51.315 INFO:teuthology.orchestra.run.smithi165.stdout:The following NEW packages will be installed: 2024-09-21T23:23:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:23:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:51.317 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages will be upgraded: 2024-09-21T23:23:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:23:51.368 INFO:teuthology.orchestra.run.smithi165.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:23:51.369 INFO:teuthology.orchestra.run.smithi165.stdout:Need to get 113 MB of archives. 2024-09-21T23:23:51.369 INFO:teuthology.orchestra.run.smithi165.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:23:51.369 INFO:teuthology.orchestra.run.smithi165.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:23:51.524 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:23:51.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:23:51.593 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:23:51.788 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-21T23:23:51.788 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-21T23:23:51.803 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:23:51.863 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:23:51.881 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:23:51.901 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:23:51.944 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:23:51.958 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:23:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:23:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:23:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:23:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:23:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: python2.7-minimal 2024-09-21T23:23:51.959 INFO:teuthology.orchestra.run.smithi079.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:23:51.960 INFO:teuthology.orchestra.run.smithi079.stdout:The following additional packages will be installed: 2024-09-21T23:23:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:23:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:51.962 INFO:teuthology.orchestra.run.smithi079.stdout:Suggested packages: 2024-09-21T23:23:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:23:51.962 INFO:teuthology.orchestra.run.smithi079.stdout:Recommended packages: 2024-09-21T23:23:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: thermald 2024-09-21T23:23:52.019 INFO:teuthology.orchestra.run.smithi079.stdout:The following NEW packages will be installed: 2024-09-21T23:23:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:23:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:52.021 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages will be upgraded: 2024-09-21T23:23:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:23:52.154 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-21T23:23:52.155 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-21T23:23:52.200 INFO:teuthology.orchestra.run.smithi079.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:23:52.200 INFO:teuthology.orchestra.run.smithi079.stdout:Need to get 113 MB of archives. 2024-09-21T23:23:52.200 INFO:teuthology.orchestra.run.smithi079.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:23:52.200 INFO:teuthology.orchestra.run.smithi079.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:23:52.342 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:23:52.342 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:23:52.342 INFO:teuthology.orchestra.run.smithi118.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:23:52.342 INFO:teuthology.orchestra.run.smithi118.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:23:52.343 INFO:teuthology.orchestra.run.smithi118.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:23:52.343 INFO:teuthology.orchestra.run.smithi118.stdout: python2.7-minimal 2024-09-21T23:23:52.343 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:23:52.344 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-09-21T23:23:52.344 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:52.344 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:23:52.345 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:52.346 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-09-21T23:23:52.346 INFO:teuthology.orchestra.run.smithi118.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:23:52.346 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-09-21T23:23:52.346 INFO:teuthology.orchestra.run.smithi118.stdout: thermald 2024-09-21T23:23:52.400 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-09-21T23:23:52.400 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:23:52.400 INFO:teuthology.orchestra.run.smithi118.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:23:52.400 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:23:52.402 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-09-21T23:23:52.402 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:23:52.454 INFO:teuthology.orchestra.run.smithi118.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:23:52.454 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 113 MB of archives. 2024-09-21T23:23:52.454 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:23:52.454 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:23:52.676 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:23:52.676 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:23:52.677 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:23:52.796 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:23:52.817 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:23:52.909 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:23:53.006 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:23:53.210 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 113 MB in 1s (75.9 MB/s) 2024-09-21T23:23:53.389 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:23:53.708 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:23:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:23:54.470 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:23:54.470 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:23:54.471 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:23:54.712 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:23:54.734 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:23:55.157 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 113 MB in 2s (48.7 MB/s) 2024-09-21T23:23:55.293 INFO:teuthology.orchestra.run.smithi165.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:23:55.297 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:23:55.334 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:23:55.436 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:23:56.265 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:23:56.266 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:23:56.266 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:23:56.658 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:23:56.704 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:23:57.107 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-21T23:23:57.277 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:23:57.347 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:23:57.351 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:23:57.406 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:23:57.426 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:23:57.506 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:23:57.513 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:23:57.983 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:23:58.004 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:23:58.041 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:23:59.307 INFO:teuthology.orchestra.run.smithi079.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:23:59.311 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:23:59.374 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:01.040 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:24:01.061 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:01.149 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:01.351 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:24:01.372 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:01.447 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:01.784 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:24:01.804 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:01.842 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:01.912 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:24:01.933 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:01.970 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:04.538 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:04.646 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:04.972 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:05.116 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:05.403 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:24:05.425 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:24:05.469 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:08.272 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:08.413 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:08.729 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:08.833 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:09.078 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:24:09.100 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:24:09.145 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:09.992 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:10.119 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:10.476 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:10.573 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:10.876 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:24:10.899 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:24:10.967 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:11.201 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:24:11.238 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:11.275 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:13.901 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:14.026 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:14.351 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:14.472 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:14.573 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:24:14.677 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:14.970 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:24:15.007 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:15.045 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:15.981 INFO:teuthology.orchestra.run.smithi165.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:15.981 INFO:teuthology.orchestra.run.smithi165.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:15.981 INFO:teuthology.orchestra.run.smithi165.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:15.981 INFO:teuthology.orchestra.run.smithi165.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:16.161 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:17.336 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:24:17.373 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:24:17.428 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:17.604 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:24:17.632 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:17.730 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:24:17.755 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:17.855 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:18.080 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:18.706 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:19.403 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:24:19.491 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:19.507 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:19.543 INFO:teuthology.orchestra.run.smithi165.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:24:19.543 INFO:teuthology.orchestra.run.smithi165.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:20.797 INFO:teuthology.orchestra.run.smithi079.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:20.797 INFO:teuthology.orchestra.run.smithi079.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:20.797 INFO:teuthology.orchestra.run.smithi079.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:20.797 INFO:teuthology.orchestra.run.smithi079.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:20.958 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:21.168 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:24:21.293 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:24:21.627 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:24:21.782 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:21.908 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:24:22.028 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:22.368 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:24:22.494 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:24:22.612 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:23.508 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:23.972 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:24.018 INFO:teuthology.orchestra.run.smithi079.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:24:24.018 INFO:teuthology.orchestra.run.smithi079.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:25.710 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:24:25.844 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:24:25.945 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:27.372 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:24:27.464 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:24:27.465 INFO:teuthology.orchestra.run.smithi118.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:33.761 INFO:teuthology.orchestra.run.smithi165.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:24:33.761 INFO:teuthology.orchestra.run.smithi165.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:24:33.771 INFO:teuthology.orchestra.run.smithi165.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:33.793 INFO:teuthology.orchestra.run.smithi165.stdout:Generating grub configuration file ... 2024-09-21T23:24:34.147 INFO:teuthology.orchestra.run.smithi165.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:34.166 INFO:teuthology.orchestra.run.smithi165.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:34.556 INFO:teuthology.orchestra.run.smithi165.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:34.563 INFO:teuthology.orchestra.run.smithi165.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:34.639 INFO:teuthology.orchestra.run.smithi165.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:34.644 INFO:teuthology.orchestra.run.smithi165.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:34.850 INFO:teuthology.orchestra.run.smithi165.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:34.850 INFO:teuthology.orchestra.run.smithi165.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:34.850 INFO:teuthology.orchestra.run.smithi165.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:34.888 INFO:teuthology.orchestra.run.smithi165.stdout:done 2024-09-21T23:24:35.347 DEBUG:teuthology.orchestra.run.smithi165:> dpkg -s linux-image-generic 2024-09-21T23:24:35.370 INFO:teuthology.orchestra.run.smithi165.stdout:Package: linux-image-generic 2024-09-21T23:24:35.370 INFO:teuthology.orchestra.run.smithi165.stdout:Status: install ok installed 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Priority: optional 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Section: kernel 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Installed-Size: 21 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: amd64 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Source: linux-meta 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Recommends: thermald 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:35.371 INFO:teuthology.orchestra.run.smithi165.stdout: available. 2024-09-21T23:24:35.372 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:24:35.372 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:24:35.372 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:24:35.372 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T23:24:35.372 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi165.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:24:35.372 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get clean 2024-09-21T23:24:35.515 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-21T23:24:35.626 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:24:35.626 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:24:35.634 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:24:35.651 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:24:37.075 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:24:37.095 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:24:37.147 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:24:37.336 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-21T23:24:37.336 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-21T23:24:37.506 INFO:teuthology.orchestra.run.smithi165.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:24:37.506 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:24:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:24:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:24:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:24:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:24:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:24:37.507 INFO:teuthology.orchestra.run.smithi165.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:24:37.546 INFO:teuthology.orchestra.run.smithi165.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:24:37.548 DEBUG:teuthology.orchestra.run.smithi165:> dpkg -s linux-image-generic 2024-09-21T23:24:37.607 INFO:teuthology.orchestra.run.smithi165.stdout:Package: linux-image-generic 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Status: install ok installed 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Priority: optional 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Section: kernel 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Installed-Size: 21 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: amd64 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Source: linux-meta 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Recommends: thermald 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:37.608 INFO:teuthology.orchestra.run.smithi165.stdout: available. 2024-09-21T23:24:37.609 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-09-21T23:24:37.653 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.EPklNsbrPx 2024-09-21T23:24:37.653 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EPklNsbrPx 2024-09-21T23:24:37.709 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.EPklNsbrPx 2024-09-21T23:24:37.780 INFO:teuthology.orchestra.run.smithi079.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:24:37.780 INFO:teuthology.orchestra.run.smithi079.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:24:37.800 INFO:teuthology.orchestra.run.smithi079.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:37.817 INFO:teuthology.orchestra.run.smithi079.stdout:Generating grub configuration file ... 2024-09-21T23:24:37.835 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.EPklNsbrPx is 10KB 2024-09-21T23:24:37.847 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.EPklNsbrPx 2024-09-21T23:24:37.852 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:37.907 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:24:37.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:24:37.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:37.972 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:24:37.972 DEBUG:teuthology.orchestra.run.smithi165:> sudo update-grub 2024-09-21T23:24:38.065 INFO:teuthology.orchestra.run.smithi079.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:38.094 INFO:teuthology.orchestra.run.smithi079.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:38.384 INFO:teuthology.orchestra.run.smithi079.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:38.390 INFO:teuthology.orchestra.run.smithi079.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout:done 2024-09-21T23:24:39.151 DEBUG:teuthology.orchestra.run.smithi079:> dpkg -s linux-image-generic 2024-09-21T23:24:39.170 INFO:teuthology.orchestra.run.smithi079.stdout:Package: linux-image-generic 2024-09-21T23:24:39.170 INFO:teuthology.orchestra.run.smithi079.stdout:Status: install ok installed 2024-09-21T23:24:39.170 INFO:teuthology.orchestra.run.smithi079.stdout:Priority: optional 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Section: kernel 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Installed-Size: 21 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: amd64 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Source: linux-meta 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Recommends: thermald 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:39.171 INFO:teuthology.orchestra.run.smithi079.stdout: available. 2024-09-21T23:24:39.172 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:24:39.172 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:24:39.172 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:24:39.172 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T23:24:39.173 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:24:39.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get clean 2024-09-21T23:24:39.269 INFO:teuthology.orchestra.run.smithi165.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:24:39.271 INFO:teuthology.orchestra.run.smithi165.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:39.274 INFO:teuthology.orchestra.run.smithi165.stderr:Generating grub configuration file ... 2024-09-21T23:24:39.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-21T23:24:39.443 INFO:teuthology.orchestra.run.smithi165.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:39.449 INFO:teuthology.orchestra.run.smithi165.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:24:39.651 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:24:39.688 INFO:teuthology.orchestra.run.smithi165.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:39.732 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:24:39.737 INFO:teuthology.orchestra.run.smithi165.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:39.742 INFO:teuthology.orchestra.run.smithi165.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:39.857 INFO:teuthology.orchestra.run.smithi165.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:39.871 INFO:teuthology.orchestra.run.smithi165.stderr:done 2024-09-21T23:24:39.886 DEBUG:teuthology.orchestra.run.smithi165:> sudo shutdown -r now 2024-09-21T23:24:41.032 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:24:41.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:24:41.105 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:24:41.289 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-21T23:24:41.290 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-21T23:24:41.454 INFO:teuthology.orchestra.run.smithi079.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:24:41.455 INFO:teuthology.orchestra.run.smithi079.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:24:41.495 INFO:teuthology.orchestra.run.smithi079.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:24:41.497 DEBUG:teuthology.orchestra.run.smithi079:> dpkg -s linux-image-generic 2024-09-21T23:24:41.556 INFO:teuthology.orchestra.run.smithi079.stdout:Package: linux-image-generic 2024-09-21T23:24:41.556 INFO:teuthology.orchestra.run.smithi079.stdout:Status: install ok installed 2024-09-21T23:24:41.556 INFO:teuthology.orchestra.run.smithi079.stdout:Priority: optional 2024-09-21T23:24:41.556 INFO:teuthology.orchestra.run.smithi079.stdout:Section: kernel 2024-09-21T23:24:41.556 INFO:teuthology.orchestra.run.smithi079.stdout:Installed-Size: 21 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: amd64 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Source: linux-meta 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Recommends: thermald 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: available. 2024-09-21T23:24:41.557 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-21T23:24:41.601 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.utPJPNswSg 2024-09-21T23:24:41.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub/grub.cfg /tmp/tmp.utPJPNswSg 2024-09-21T23:24:41.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.utPJPNswSg 2024-09-21T23:24:41.783 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.utPJPNswSg is 10KB 2024-09-21T23:24:41.796 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.utPJPNswSg 2024-09-21T23:24:41.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:41.852 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:24:41.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:24:41.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:41.916 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:24:41.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo update-grub 2024-09-21T23:24:43.158 INFO:teuthology.orchestra.run.smithi079.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:24:43.159 INFO:teuthology.orchestra.run.smithi079.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:43.171 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-09-21T23:24:43.343 INFO:teuthology.orchestra.run.smithi079.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:43.349 INFO:teuthology.orchestra.run.smithi079.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:43.592 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:24:43.592 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:24:43.605 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:43.614 INFO:teuthology.orchestra.run.smithi079.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:43.617 INFO:teuthology.orchestra.run.smithi118.stdout:Generating grub configuration file ... 2024-09-21T23:24:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:43.726 INFO:teuthology.orchestra.run.smithi079.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:43.726 INFO:teuthology.orchestra.run.smithi079.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:43.726 INFO:teuthology.orchestra.run.smithi079.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:43.737 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-09-21T23:24:43.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-09-21T23:24:43.919 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:43.935 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:44.248 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:44.254 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:44.323 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:44.330 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:44.492 INFO:teuthology.orchestra.run.smithi118.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:44.492 INFO:teuthology.orchestra.run.smithi118.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:44.492 INFO:teuthology.orchestra.run.smithi118.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:44.533 INFO:teuthology.orchestra.run.smithi118.stdout:done 2024-09-21T23:24:45.020 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 21 2024-09-21T23:24:45.085 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:45.086 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-09-21T23:24:45.086 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:24:45.086 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:24:45.086 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:24:45.087 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T23:24:45.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:24:45.087 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-09-21T23:24:45.269 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-21T23:24:45.384 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:24:45.507 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:24:45.596 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:24:45.669 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:24:46.861 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:24:46.883 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:24:46.946 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:24:47.160 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-21T23:24:47.160 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-21T23:24:47.351 INFO:teuthology.orchestra.run.smithi118.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:24:47.351 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:24:47.352 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:24:47.352 INFO:teuthology.orchestra.run.smithi118.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:24:47.352 INFO:teuthology.orchestra.run.smithi118.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:24:47.352 INFO:teuthology.orchestra.run.smithi118.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:24:47.353 INFO:teuthology.orchestra.run.smithi118.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:24:47.353 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:24:47.392 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:24:47.393 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-09-21T23:24:47.408 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 21 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.15.0.122.122 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-09-21T23:24:47.409 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:24:47.410 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-09-21T23:24:47.410 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-21T23:24:47.455 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.K6ipdetu4M 2024-09-21T23:24:47.455 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub/grub.cfg /tmp/tmp.K6ipdetu4M 2024-09-21T23:24:47.510 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.K6ipdetu4M 2024-09-21T23:24:47.628 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.K6ipdetu4M is 10KB 2024-09-21T23:24:47.641 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.K6ipdetu4M 2024-09-21T23:24:47.646 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:47.698 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:24:47.698 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:24:47.698 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:24:47.777 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:24:47.777 DEBUG:teuthology.orchestra.run.smithi118:> sudo update-grub 2024-09-21T23:24:49.975 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:24:49.976 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:24:49.978 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2024-09-21T23:24:50.184 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:24:50.190 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:24:50.470 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:24:50.475 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:24:50.516 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:24:50.521 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:24:50.663 INFO:teuthology.orchestra.run.smithi118.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:24:50.663 INFO:teuthology.orchestra.run.smithi118.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:24:50.663 INFO:teuthology.orchestra.run.smithi118.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:24:50.683 INFO:teuthology.orchestra.run.smithi118.stderr:done 2024-09-21T23:24:50.685 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2024-09-21T23:25:09.906 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T23:25:09.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:25:09.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:13.758 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T23:25:13.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:25:13.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:20.692 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T23:25:20.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:25:20.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:28.431 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-21T23:25:32.271 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T23:25:37.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:25:37.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:39.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-21T23:25:40.495 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-21T23:25:41.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:25:41.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:44.335 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T23:25:48.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:25:48.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:51.247 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-21T23:25:52.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:25:52.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:25:56.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:25:56.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:03.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:26:03.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:06.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-21T23:26:10.929 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-21T23:26:14.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T23:26:21.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:26:21.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:25.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:26:25.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:26.595 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-21T23:26:27.580 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:26:27.580 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.15.0-122-generic"... 2024-09-21T23:26:27.580 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-21T23:26:27.626 INFO:teuthology.orchestra.run.smithi165.stdout:5.15.0-122-generic 2024-09-21T23:26:27.626 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:26:27.626 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:26:27.626 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:26:28.626 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-21T23:26:28.627 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:26:28.765 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-09-21T23:26:28.795 DEBUG:teuthology.parallel:result is None 2024-09-21T23:26:29.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:26:29.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:29.769 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-21T23:26:39.855 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-21T23:26:47.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:26:47.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:48.076 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-21T23:26:48.572 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:26:48.572 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T23:26:48.572 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-21T23:26:48.619 INFO:teuthology.orchestra.run.smithi079.stdout:5.15.0-122-generic 2024-09-21T23:26:48.619 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:26:48.619 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:26:48.619 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:26:49.621 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T23:26:49.621 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:26:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-09-21T23:26:49.717 DEBUG:teuthology.parallel:result is None 2024-09-21T23:26:57.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:26:57.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:26:58.131 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-21T23:26:58.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:26:58.655 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T23:26:58.655 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-21T23:26:58.703 INFO:teuthology.orchestra.run.smithi118.stdout:5.15.0-122-generic 2024-09-21T23:26:58.703 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:26:58.703 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:26:58.703 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:26:59.704 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T23:26:59.705 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:26:59.786 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2024-09-21T23:26:59.817 DEBUG:teuthology.parallel:result is None 2024-09-21T23:26:59.817 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T23:26:59.836 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T23:26:59.837 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:26:59.839 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:26:59.841 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:26:59.847 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T23:26:59.904 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T23:26:59.937 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T23:26:59.938 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:26:59.940 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:26:59.942 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:27:00.196 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T23:27:00.262 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T23:27:00.262 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:27:00.265 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:27:00.267 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:27:00.284 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.292 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.293 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.297 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.302 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.307 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:27:00.308 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T23:27:00.315 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T23:27:00.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:27:00.336 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:27:00.348 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:27:00.361 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T23:27:00.370 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T23:27:00.370 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:27:00.392 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:27:00.408 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:27:00.414 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:27:00.443 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:27:00.491 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:27:00.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:27:00.556 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:27:00.562 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:27:00.607 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:27:00.608 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:27:00.675 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:27:00.681 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:27:00.730 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:27:00.730 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:27:00.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-09-21T23:27:00.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-09-21T23:27:00.803 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-09-21T23:27:00.871 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T23:27:00.885 INFO:teuthology.task.internal:Starting timer... 2024-09-21T23:27:00.885 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T23:27:00.950 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T23:27:00.982 DEBUG:teuthology.task.selinux:Excluding smithi079: OS 'ubuntu' does not support SELinux 2024-09-21T23:27:00.982 DEBUG:teuthology.task.selinux:Excluding smithi118: OS 'ubuntu' does not support SELinux 2024-09-21T23:27:00.982 DEBUG:teuthology.task.selinux:Excluding smithi165: OS 'ubuntu' does not support SELinux 2024-09-21T23:27:00.982 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T23:27:00.982 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T23:27:00.982 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T23:27:00.982 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T23:27:01.015 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T23:27:01.026 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T23:27:01.027 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi118.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T23:34:33.360 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-09-21T23:34:33.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:34:33.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:34:33.443 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-21T23:34:33.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-21T23:34:33.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:34:33.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:34:33.591 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-21T23:34:33.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-21T23:34:33.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:34:33.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:34:33.751 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-21T23:34:33.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T23:34:33.832 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T23:34:33.841 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T23:34:33.841 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:34:33.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:34:33.844 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:34:33.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:34:33.846 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:34:33.846 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Command line: ntpd -gq 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: ---------------------------------------------------- 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: corporation. Support and training for ntp-4 are 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: available at https://www.nwtime.org/support 2024-09-21T23:34:33.871 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: ---------------------------------------------------- 2024-09-21T23:34:33.872 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: proto: precision = 0.045 usec (-24) 2024-09-21T23:34:33.872 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: basedate set to 2022-02-04 2024-09-21T23:34:33.872 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:34:33.872 INFO:teuthology.orchestra.run.smithi079.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.873 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.873 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.873 INFO:teuthology.orchestra.run.smithi079.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen normally on 3 enp3s0f1 172.21.15.79:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen normally on 4 lo [::1]:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce87%5]:123 2024-09-21T23:34:33.874 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:33 ntpd[18491]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Command line: ntpd -gq 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: ---------------------------------------------------- 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: corporation. Support and training for ntp-4 are 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: available at https://www.nwtime.org/support 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: ---------------------------------------------------- 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: proto: precision = 0.045 usec (-24) 2024-09-21T23:34:33.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: basedate set to 2022-02-04 2024-09-21T23:34:33.876 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:34:33.876 INFO:teuthology.orchestra.run.smithi118.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.876 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.876 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.876 INFO:teuthology.orchestra.run.smithi118.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.877 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:34:33.877 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:34:33.877 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:34:33.877 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen normally on 3 enp3s0f1 172.21.15.118:123 2024-09-21T23:34:33.877 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen normally on 4 lo [::1]:123 2024-09-21T23:34:33.878 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d725%5]:123 2024-09-21T23:34:33.878 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:33 ntpd[18453]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Command line: ntpd -gq 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: ---------------------------------------------------- 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: corporation. Support and training for ntp-4 are 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: available at https://www.nwtime.org/support 2024-09-21T23:34:33.904 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: ---------------------------------------------------- 2024-09-21T23:34:33.905 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: proto: precision = 0.058 usec (-24) 2024-09-21T23:34:33.905 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: basedate set to 2022-02-04 2024-09-21T23:34:33.905 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:34:33.905 INFO:teuthology.orchestra.run.smithi165.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.905 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:34:33.906 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.906 INFO:teuthology.orchestra.run.smithi165.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen normally on 3 enp3s0f1 172.21.15.165:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen normally on 4 lo [::1]:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f63%5]:123 2024-09-21T23:34:33.907 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:33 ntpd[18456]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:34:34.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:34.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:34 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:34.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:34.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:34.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:34 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:34.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:34 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:34.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:34 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:35 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:35.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:35 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:36 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:36 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:36 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:37 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:37.871 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:37 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:37.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:37.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:37.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:37.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:37.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:37.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:37 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:37.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:37 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.874 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:38 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:38 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:38.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:38 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:39.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:39.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:39.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:39.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:39.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:39.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:39 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:39.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:39.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:39 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:40.872 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:40.873 INFO:teuthology.orchestra.run.smithi079.stdout:21 Sep 23:34:40 ntpd[18491]: ntpd: time slew -0.002379 s 2024-09-21T23:34:40.873 INFO:teuthology.orchestra.run.smithi079.stdout:ntpd: time slew -0.002379s 2024-09-21T23:34:40.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:34:40.873 INFO:teuthology.orchestra.run.smithi079.stderr:21 Sep 23:34:40 ntpd[18491]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:34:40.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.874 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stdout:21 Sep 23:34:40 ntpd[18453]: ntpd: time slew +0.002741 s 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stdout:ntpd: time slew +0.002741s 2024-09-21T23:34:40.875 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:34:40.876 INFO:teuthology.orchestra.run.smithi118.stderr:21 Sep 23:34:40 ntpd[18453]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:40 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.932 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================== 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stdout:21 Sep 23:34:41 ntpd[18456]: ntpd: time slew +0.003137 s 2024-09-21T23:34:41.905 INFO:teuthology.orchestra.run.smithi165.stdout:ntpd: time slew +0.003137s 2024-09-21T23:34:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:34:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:21 Sep 23:34:41 ntpd[18456]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:34:41.961 INFO:teuthology.orchestra.run.smithi165.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:34:41.962 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================== 2024-09-21T23:34:41.962 INFO:teuthology.orchestra.run.smithi165.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:41.962 INFO:teuthology.orchestra.run.smithi165.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:41.962 INFO:teuthology.orchestra.run.smithi165.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:41.962 INFO:teuthology.orchestra.run.smithi165.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:34:41.963 INFO:teuthology.run_tasks:Running task install... 2024-09-21T23:34:41.976 DEBUG:teuthology.task.install:project ceph 2024-09-21T23:34:41.976 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:34:41.976 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-21T23:34:41.976 INFO:teuthology.task.install:Using flavor: default 2024-09-21T23:34:41.981 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T23:34:41.981 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-21T23:34:41.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-key list | grep Ceph 2024-09-21T23:34:41.982 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-key list | grep Ceph 2024-09-21T23:34:41.982 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-key list | grep Ceph 2024-09-21T23:34:42.046 INFO:teuthology.orchestra.run.smithi079.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:34:42.048 INFO:teuthology.orchestra.run.smithi118.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:34:42.063 INFO:teuthology.orchestra.run.smithi165.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:34:42.077 INFO:teuthology.orchestra.run.smithi079.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:34:42.077 INFO:teuthology.orchestra.run.smithi079.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:34:42.078 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:34:42.079 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:34:42.083 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:34:42.083 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:34:42.083 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:34:42.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:34:42.090 INFO:teuthology.orchestra.run.smithi165.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:34:42.090 INFO:teuthology.orchestra.run.smithi165.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:34:42.091 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:34:42.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:34:42.220 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:34:42.220 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:42.228 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:34:42.228 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:42.232 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:34:42.232 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:42.329 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:34:42.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:34:42.337 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:34:42.337 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:34:42.344 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:34:42.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:34:42.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-21T23:34:42.351 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-21T23:34:42.359 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-21T23:34:42.513 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:34:42.519 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:34:42.527 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:34:42.529 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:34:42.537 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:34:42.546 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:34:42.554 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:34:42.566 INFO:teuthology.orchestra.run.smithi079.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:34:42.575 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:34:42.582 INFO:teuthology.orchestra.run.smithi165.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:34:42.587 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:34:42.597 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:34:42.603 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:34:42.609 INFO:teuthology.orchestra.run.smithi079.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:34:42.619 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:34:42.626 INFO:teuthology.orchestra.run.smithi165.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:34:42.630 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:34:42.635 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:34:42.640 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:34:42.645 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:34:42.646 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:34:42.652 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:34:42.657 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:34:42.668 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:34:42.674 INFO:teuthology.orchestra.run.smithi165.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:34:42.719 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:34:42.797 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:34:42.868 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 28.9 kB in 0s (85.4 kB/s) 2024-09-21T23:34:42.882 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 28.9 kB in 0s (84.8 kB/s) 2024-09-21T23:34:42.948 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 28.9 kB in 0s (69.6 kB/s) 2024-09-21T23:34:43.862 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:34:43.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:43.901 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:34:43.916 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-21T23:34:43.969 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-21T23:34:44.013 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:34:44.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:34:44.092 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-21T23:34:44.153 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-21T23:34:44.154 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-21T23:34:44.155 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-21T23:34:44.156 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-21T23:34:44.304 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-21T23:34:44.305 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-21T23:34:44.343 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:34:44.343 INFO:teuthology.orchestra.run.smithi165.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:34:44.343 INFO:teuthology.orchestra.run.smithi165.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:34:44.344 INFO:teuthology.orchestra.run.smithi165.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:34:44.344 INFO:teuthology.orchestra.run.smithi165.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:34:44.344 INFO:teuthology.orchestra.run.smithi165.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:34:44.344 INFO:teuthology.orchestra.run.smithi165.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:34:44.345 INFO:teuthology.orchestra.run.smithi165.stdout:The following additional packages will be installed: 2024-09-21T23:34:44.345 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:34:44.345 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:34:44.345 INFO:teuthology.orchestra.run.smithi165.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:34:44.345 INFO:teuthology.orchestra.run.smithi165.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:34:44.346 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:34:44.346 INFO:teuthology.orchestra.run.smithi165.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:34:44.347 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet zip 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout:Suggested packages: 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:34:44.348 INFO:teuthology.orchestra.run.smithi165.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:34:44.349 INFO:teuthology.orchestra.run.smithi165.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:34:44.349 INFO:teuthology.orchestra.run.smithi165.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:34:44.349 INFO:teuthology.orchestra.run.smithi165.stdout:Recommended packages: 2024-09-21T23:34:44.349 INFO:teuthology.orchestra.run.smithi165.stdout: btrfs-tools 2024-09-21T23:34:44.366 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:34:44.367 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:34:44.368 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-09-21T23:34:44.368 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:34:44.368 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:34:44.369 INFO:teuthology.orchestra.run.smithi118.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:34:44.369 INFO:teuthology.orchestra.run.smithi118.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:34:44.369 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:34:44.370 INFO:teuthology.orchestra.run.smithi118.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet zip 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:34:44.371 INFO:teuthology.orchestra.run.smithi118.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:34:44.372 INFO:teuthology.orchestra.run.smithi118.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:34:44.372 INFO:teuthology.orchestra.run.smithi118.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:34:44.372 INFO:teuthology.orchestra.run.smithi118.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:34:44.372 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-09-21T23:34:44.372 INFO:teuthology.orchestra.run.smithi118.stdout: btrfs-tools 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout:The following NEW packages will be installed: 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:34:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:34:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:34:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:34:44.475 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-09-21T23:34:44.475 INFO:teuthology.orchestra.run.smithi118.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:34:44.476 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:34:44.476 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:34:44.476 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.477 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:34:44.478 INFO:teuthology.orchestra.run.smithi118.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:34:44.503 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:34:44.503 INFO:teuthology.orchestra.run.smithi079.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:34:44.503 INFO:teuthology.orchestra.run.smithi079.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:34:44.503 INFO:teuthology.orchestra.run.smithi079.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:34:44.504 INFO:teuthology.orchestra.run.smithi079.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:34:44.504 INFO:teuthology.orchestra.run.smithi079.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:34:44.504 INFO:teuthology.orchestra.run.smithi079.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:34:44.505 INFO:teuthology.orchestra.run.smithi079.stdout:The following additional packages will be installed: 2024-09-21T23:34:44.505 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:34:44.505 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:34:44.505 INFO:teuthology.orchestra.run.smithi079.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:34:44.505 INFO:teuthology.orchestra.run.smithi079.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:34:44.506 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:34:44.506 INFO:teuthology.orchestra.run.smithi079.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:34:44.506 INFO:teuthology.orchestra.run.smithi079.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:34:44.506 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:34:44.507 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet zip 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout:Suggested packages: 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout:Recommended packages: 2024-09-21T23:34:44.508 INFO:teuthology.orchestra.run.smithi079.stdout: btrfs-tools 2024-09-21T23:34:44.511 INFO:teuthology.orchestra.run.smithi165.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:34:44.512 INFO:teuthology.orchestra.run.smithi165.stdout:Need to get 181 MB of archives. 2024-09-21T23:34:44.512 INFO:teuthology.orchestra.run.smithi165.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:34:44.512 INFO:teuthology.orchestra.run.smithi165.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:34:44.538 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:34:44.539 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 181 MB of archives. 2024-09-21T23:34:44.539 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:34:44.539 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:34:44.594 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:34:44.601 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:34:44.603 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:34:44.610 INFO:teuthology.orchestra.run.smithi079.stdout:The following NEW packages will be installed: 2024-09-21T23:34:44.610 INFO:teuthology.orchestra.run.smithi079.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:34:44.610 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:34:44.611 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:34:44.612 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:34:44.613 INFO:teuthology.orchestra.run.smithi079.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:34:44.621 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:34:44.623 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:34:44.627 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:34:44.630 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:34:44.647 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:34:44.672 INFO:teuthology.orchestra.run.smithi079.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:34:44.672 INFO:teuthology.orchestra.run.smithi079.stdout:Need to get 181 MB of archives. 2024-09-21T23:34:44.672 INFO:teuthology.orchestra.run.smithi079.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:34:44.672 INFO:teuthology.orchestra.run.smithi079.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:34:44.680 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:34:44.685 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:34:44.697 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:34:44.701 INFO:teuthology.orchestra.run.smithi165.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:34:44.702 INFO:teuthology.orchestra.run.smithi165.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:34:44.703 INFO:teuthology.orchestra.run.smithi165.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:34:44.707 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:34:44.711 INFO:teuthology.orchestra.run.smithi165.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:34:44.712 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:34:44.713 INFO:teuthology.orchestra.run.smithi165.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:34:44.716 INFO:teuthology.orchestra.run.smithi165.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:34:44.717 INFO:teuthology.orchestra.run.smithi165.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:34:44.720 INFO:teuthology.orchestra.run.smithi165.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:34:44.723 INFO:teuthology.orchestra.run.smithi165.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:34:44.724 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:34:44.728 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:34:44.729 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:34:44.730 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:34:44.735 INFO:teuthology.orchestra.run.smithi165.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:34:44.737 INFO:teuthology.orchestra.run.smithi165.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:34:44.738 INFO:teuthology.orchestra.run.smithi165.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:34:44.738 INFO:teuthology.orchestra.run.smithi165.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:34:44.738 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:34:44.739 INFO:teuthology.orchestra.run.smithi165.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:34:44.739 INFO:teuthology.orchestra.run.smithi165.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:34:44.740 INFO:teuthology.orchestra.run.smithi165.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:34:44.740 INFO:teuthology.orchestra.run.smithi165.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:34:44.741 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:34:44.743 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:34:44.744 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:34:44.746 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:34:44.748 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:34:44.755 INFO:teuthology.orchestra.run.smithi165.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:34:44.755 INFO:teuthology.orchestra.run.smithi165.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:34:44.762 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:34:44.763 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:34:44.764 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:34:44.764 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:34:44.765 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:34:44.766 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:34:44.766 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:34:44.767 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:34:44.767 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:34:44.767 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:34:44.772 INFO:teuthology.orchestra.run.smithi165.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:34:44.775 INFO:teuthology.orchestra.run.smithi165.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:34:44.775 INFO:teuthology.orchestra.run.smithi165.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:34:44.776 INFO:teuthology.orchestra.run.smithi165.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:34:44.776 INFO:teuthology.orchestra.run.smithi165.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:34:44.777 INFO:teuthology.orchestra.run.smithi165.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:34:44.778 INFO:teuthology.orchestra.run.smithi165.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:34:44.778 INFO:teuthology.orchestra.run.smithi165.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:34:44.780 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:34:44.780 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:34:44.782 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:34:44.783 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:34:44.786 INFO:teuthology.orchestra.run.smithi165.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:34:44.790 INFO:teuthology.orchestra.run.smithi165.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:34:44.790 INFO:teuthology.orchestra.run.smithi165.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:34:44.797 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:34:44.799 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:34:44.800 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:34:44.800 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:34:44.800 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:34:44.801 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:34:44.802 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:34:44.802 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:34:44.807 INFO:teuthology.orchestra.run.smithi165.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:34:44.813 INFO:teuthology.orchestra.run.smithi165.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:34:44.813 INFO:teuthology.orchestra.run.smithi165.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:34:44.813 INFO:teuthology.orchestra.run.smithi165.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:34:44.814 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:34:44.814 INFO:teuthology.orchestra.run.smithi165.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:34:44.815 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:34:44.815 INFO:teuthology.orchestra.run.smithi165.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:34:44.817 INFO:teuthology.orchestra.run.smithi165.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:34:44.818 INFO:teuthology.orchestra.run.smithi165.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:34:44.824 INFO:teuthology.orchestra.run.smithi165.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:34:44.825 INFO:teuthology.orchestra.run.smithi165.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:34:44.831 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:34:44.836 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:34:44.836 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:34:44.837 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:34:44.837 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:34:44.838 INFO:teuthology.orchestra.run.smithi118.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:34:44.840 INFO:teuthology.orchestra.run.smithi118.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:34:44.841 INFO:teuthology.orchestra.run.smithi118.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:34:44.842 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:34:44.842 INFO:teuthology.orchestra.run.smithi165.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:34:44.842 INFO:teuthology.orchestra.run.smithi165.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:34:44.843 INFO:teuthology.orchestra.run.smithi165.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:34:44.846 INFO:teuthology.orchestra.run.smithi165.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:34:44.846 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:34:44.848 INFO:teuthology.orchestra.run.smithi118.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:34:44.849 INFO:teuthology.orchestra.run.smithi165.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:34:44.849 INFO:teuthology.orchestra.run.smithi118.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:34:44.858 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:34:44.860 INFO:teuthology.orchestra.run.smithi165.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:34:44.862 INFO:teuthology.orchestra.run.smithi079.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:34:44.863 INFO:teuthology.orchestra.run.smithi079.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:34:44.864 INFO:teuthology.orchestra.run.smithi079.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:34:44.865 INFO:teuthology.orchestra.run.smithi165.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:34:44.866 INFO:teuthology.orchestra.run.smithi165.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:34:44.866 INFO:teuthology.orchestra.run.smithi118.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:34:44.867 INFO:teuthology.orchestra.run.smithi118.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:34:44.868 INFO:teuthology.orchestra.run.smithi118.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:34:44.869 INFO:teuthology.orchestra.run.smithi165.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:34:44.870 INFO:teuthology.orchestra.run.smithi118.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:34:44.871 INFO:teuthology.orchestra.run.smithi165.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:34:44.872 INFO:teuthology.orchestra.run.smithi079.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:34:44.874 INFO:teuthology.orchestra.run.smithi079.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:34:44.876 INFO:teuthology.orchestra.run.smithi165.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:34:44.877 INFO:teuthology.orchestra.run.smithi079.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:34:44.878 INFO:teuthology.orchestra.run.smithi079.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:34:44.880 INFO:teuthology.orchestra.run.smithi079.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:34:44.880 INFO:teuthology.orchestra.run.smithi118.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:34:44.882 INFO:teuthology.orchestra.run.smithi079.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:34:44.896 INFO:teuthology.orchestra.run.smithi079.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:34:44.898 INFO:teuthology.orchestra.run.smithi079.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:34:44.898 INFO:teuthology.orchestra.run.smithi079.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:34:44.898 INFO:teuthology.orchestra.run.smithi079.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:34:44.900 INFO:teuthology.orchestra.run.smithi079.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:34:44.900 INFO:teuthology.orchestra.run.smithi079.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:34:44.900 INFO:teuthology.orchestra.run.smithi079.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:34:44.901 INFO:teuthology.orchestra.run.smithi079.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:34:44.912 INFO:teuthology.orchestra.run.smithi165.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:34:44.913 INFO:teuthology.orchestra.run.smithi165.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:34:44.915 INFO:teuthology.orchestra.run.smithi165.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:34:44.915 INFO:teuthology.orchestra.run.smithi165.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:34:44.915 INFO:teuthology.orchestra.run.smithi079.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:34:44.915 INFO:teuthology.orchestra.run.smithi079.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:34:44.916 INFO:teuthology.orchestra.run.smithi165.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:34:44.922 INFO:teuthology.orchestra.run.smithi118.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:34:44.923 INFO:teuthology.orchestra.run.smithi118.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:34:44.924 INFO:teuthology.orchestra.run.smithi118.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:34:44.924 INFO:teuthology.orchestra.run.smithi118.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:34:44.925 INFO:teuthology.orchestra.run.smithi118.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:34:44.931 INFO:teuthology.orchestra.run.smithi079.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:34:44.934 INFO:teuthology.orchestra.run.smithi079.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:34:44.935 INFO:teuthology.orchestra.run.smithi079.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:34:44.935 INFO:teuthology.orchestra.run.smithi079.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:34:44.935 INFO:teuthology.orchestra.run.smithi079.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:34:44.936 INFO:teuthology.orchestra.run.smithi079.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:34:44.937 INFO:teuthology.orchestra.run.smithi079.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:34:44.937 INFO:teuthology.orchestra.run.smithi079.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:34:44.940 INFO:teuthology.orchestra.run.smithi165.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:34:44.942 INFO:teuthology.orchestra.run.smithi118.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:34:44.944 INFO:teuthology.orchestra.run.smithi118.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:34:44.944 INFO:teuthology.orchestra.run.smithi118.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:34:44.945 INFO:teuthology.orchestra.run.smithi165.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:34:44.948 INFO:teuthology.orchestra.run.smithi165.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:34:44.948 INFO:teuthology.orchestra.run.smithi079.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:34:44.949 INFO:teuthology.orchestra.run.smithi079.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:34:44.960 INFO:teuthology.orchestra.run.smithi118.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:34:44.966 INFO:teuthology.orchestra.run.smithi165.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:34:44.966 INFO:teuthology.orchestra.run.smithi165.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:34:44.966 INFO:teuthology.orchestra.run.smithi165.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:34:44.967 INFO:teuthology.orchestra.run.smithi165.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:34:44.967 INFO:teuthology.orchestra.run.smithi165.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:34:44.967 INFO:teuthology.orchestra.run.smithi165.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:34:44.967 INFO:teuthology.orchestra.run.smithi079.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:34:44.969 INFO:teuthology.orchestra.run.smithi165.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:34:44.970 INFO:teuthology.orchestra.run.smithi165.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:34:44.971 INFO:teuthology.orchestra.run.smithi079.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:34:44.971 INFO:teuthology.orchestra.run.smithi079.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:34:44.972 INFO:teuthology.orchestra.run.smithi079.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:34:44.973 INFO:teuthology.orchestra.run.smithi079.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:34:44.974 INFO:teuthology.orchestra.run.smithi165.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:34:44.974 INFO:teuthology.orchestra.run.smithi079.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:34:44.974 INFO:teuthology.orchestra.run.smithi118.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:34:44.975 INFO:teuthology.orchestra.run.smithi118.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:34:44.975 INFO:teuthology.orchestra.run.smithi079.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:34:44.977 INFO:teuthology.orchestra.run.smithi079.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:34:44.978 INFO:teuthology.orchestra.run.smithi118.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:34:44.983 INFO:teuthology.orchestra.run.smithi079.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:34:44.984 INFO:teuthology.orchestra.run.smithi079.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:34:44.987 INFO:teuthology.orchestra.run.smithi165.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:34:44.988 INFO:teuthology.orchestra.run.smithi165.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:34:44.989 INFO:teuthology.orchestra.run.smithi165.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:34:44.992 INFO:teuthology.orchestra.run.smithi118.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:34:44.992 INFO:teuthology.orchestra.run.smithi165.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:34:44.993 INFO:teuthology.orchestra.run.smithi118.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:34:44.994 INFO:teuthology.orchestra.run.smithi118.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:34:44.995 INFO:teuthology.orchestra.run.smithi165.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:34:44.995 INFO:teuthology.orchestra.run.smithi165.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:34:44.996 INFO:teuthology.orchestra.run.smithi165.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:34:44.997 INFO:teuthology.orchestra.run.smithi118.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:34:44.999 INFO:teuthology.orchestra.run.smithi118.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:34:44.999 INFO:teuthology.orchestra.run.smithi118.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:34:45.000 INFO:teuthology.orchestra.run.smithi118.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:34:45.000 INFO:teuthology.orchestra.run.smithi079.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:34:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:34:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:34:45.004 INFO:teuthology.orchestra.run.smithi165.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:34:45.004 INFO:teuthology.orchestra.run.smithi079.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:34:45.005 INFO:teuthology.orchestra.run.smithi165.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:34:45.007 INFO:teuthology.orchestra.run.smithi165.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:34:45.009 INFO:teuthology.orchestra.run.smithi118.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:34:45.010 INFO:teuthology.orchestra.run.smithi118.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:34:45.012 INFO:teuthology.orchestra.run.smithi118.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:34:45.021 INFO:teuthology.orchestra.run.smithi165.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:34:45.022 INFO:teuthology.orchestra.run.smithi165.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:34:45.022 INFO:teuthology.orchestra.run.smithi165.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:34:45.024 INFO:teuthology.orchestra.run.smithi165.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:34:45.025 INFO:teuthology.orchestra.run.smithi165.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:34:45.027 INFO:teuthology.orchestra.run.smithi118.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:34:45.028 INFO:teuthology.orchestra.run.smithi118.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:34:45.028 INFO:teuthology.orchestra.run.smithi118.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:34:45.029 INFO:teuthology.orchestra.run.smithi118.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:34:45.029 INFO:teuthology.orchestra.run.smithi118.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:34:45.030 INFO:teuthology.orchestra.run.smithi118.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:34:45.031 INFO:teuthology.orchestra.run.smithi165.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:34:45.032 INFO:teuthology.orchestra.run.smithi165.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:34:45.035 INFO:teuthology.orchestra.run.smithi118.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:34:45.036 INFO:teuthology.orchestra.run.smithi118.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:34:45.039 INFO:teuthology.orchestra.run.smithi165.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:34:45.042 INFO:teuthology.orchestra.run.smithi165.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:34:45.042 INFO:teuthology.orchestra.run.smithi165.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:34:45.044 INFO:teuthology.orchestra.run.smithi118.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:34:45.044 INFO:teuthology.orchestra.run.smithi118.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:34:45.047 INFO:teuthology.orchestra.run.smithi118.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:34:45.047 INFO:teuthology.orchestra.run.smithi118.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:34:45.051 INFO:teuthology.orchestra.run.smithi118.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:34:45.052 INFO:teuthology.orchestra.run.smithi118.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:34:45.056 INFO:teuthology.orchestra.run.smithi118.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:34:45.057 INFO:teuthology.orchestra.run.smithi118.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:34:45.062 INFO:teuthology.orchestra.run.smithi079.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:34:45.063 INFO:teuthology.orchestra.run.smithi079.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:34:45.065 INFO:teuthology.orchestra.run.smithi079.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:34:45.065 INFO:teuthology.orchestra.run.smithi079.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:34:45.066 INFO:teuthology.orchestra.run.smithi079.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:34:45.083 INFO:teuthology.orchestra.run.smithi079.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:34:45.085 INFO:teuthology.orchestra.run.smithi079.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:34:45.085 INFO:teuthology.orchestra.run.smithi079.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:34:45.094 INFO:teuthology.orchestra.run.smithi165.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:34:45.097 INFO:teuthology.orchestra.run.smithi165.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:34:45.100 INFO:teuthology.orchestra.run.smithi079.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:34:45.100 INFO:teuthology.orchestra.run.smithi079.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:34:45.101 INFO:teuthology.orchestra.run.smithi079.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:34:45.101 INFO:teuthology.orchestra.run.smithi079.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:34:45.102 INFO:teuthology.orchestra.run.smithi079.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:34:45.102 INFO:teuthology.orchestra.run.smithi079.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:34:45.105 INFO:teuthology.orchestra.run.smithi165.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:34:45.107 INFO:teuthology.orchestra.run.smithi118.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:34:45.116 INFO:teuthology.orchestra.run.smithi079.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:34:45.116 INFO:teuthology.orchestra.run.smithi079.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:34:45.119 INFO:teuthology.orchestra.run.smithi079.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:34:45.133 INFO:teuthology.orchestra.run.smithi079.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:34:45.134 INFO:teuthology.orchestra.run.smithi079.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:34:45.135 INFO:teuthology.orchestra.run.smithi079.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:34:45.138 INFO:teuthology.orchestra.run.smithi079.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:34:45.141 INFO:teuthology.orchestra.run.smithi079.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:34:45.141 INFO:teuthology.orchestra.run.smithi079.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:34:45.142 INFO:teuthology.orchestra.run.smithi079.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:34:45.151 INFO:teuthology.orchestra.run.smithi079.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:34:45.151 INFO:teuthology.orchestra.run.smithi079.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:34:45.154 INFO:teuthology.orchestra.run.smithi079.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:34:45.168 INFO:teuthology.orchestra.run.smithi079.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:34:45.168 INFO:teuthology.orchestra.run.smithi079.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:34:45.169 INFO:teuthology.orchestra.run.smithi079.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:34:45.170 INFO:teuthology.orchestra.run.smithi079.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:34:45.171 INFO:teuthology.orchestra.run.smithi079.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:34:45.177 INFO:teuthology.orchestra.run.smithi079.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:34:45.177 INFO:teuthology.orchestra.run.smithi079.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:34:45.185 INFO:teuthology.orchestra.run.smithi079.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:34:45.187 INFO:teuthology.orchestra.run.smithi079.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:34:45.187 INFO:teuthology.orchestra.run.smithi079.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:34:45.337 INFO:teuthology.orchestra.run.smithi118.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:34:45.340 INFO:teuthology.orchestra.run.smithi118.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:34:45.349 INFO:teuthology.orchestra.run.smithi118.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:34:45.960 INFO:teuthology.orchestra.run.smithi079.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:34:45.970 INFO:teuthology.orchestra.run.smithi165.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:34:46.058 INFO:teuthology.orchestra.run.smithi165.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:34:46.062 INFO:teuthology.orchestra.run.smithi165.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:34:46.065 INFO:teuthology.orchestra.run.smithi165.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:34:46.105 INFO:teuthology.orchestra.run.smithi165.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:34:46.298 INFO:teuthology.orchestra.run.smithi165.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:34:46.465 INFO:teuthology.orchestra.run.smithi079.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:34:46.543 INFO:teuthology.orchestra.run.smithi079.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:34:46.565 INFO:teuthology.orchestra.run.smithi118.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:34:46.611 INFO:teuthology.orchestra.run.smithi079.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:34:46.612 INFO:teuthology.orchestra.run.smithi079.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:34:46.679 INFO:teuthology.orchestra.run.smithi079.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:34:46.681 INFO:teuthology.orchestra.run.smithi079.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:34:46.688 INFO:teuthology.orchestra.run.smithi079.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:34:46.753 INFO:teuthology.orchestra.run.smithi118.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:34:46.779 INFO:teuthology.orchestra.run.smithi118.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:34:46.781 INFO:teuthology.orchestra.run.smithi118.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:34:46.859 INFO:teuthology.orchestra.run.smithi118.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:34:47.045 INFO:teuthology.orchestra.run.smithi165.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:34:47.073 INFO:teuthology.orchestra.run.smithi165.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:34:47.090 INFO:teuthology.orchestra.run.smithi165.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:34:47.117 INFO:teuthology.orchestra.run.smithi118.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:34:47.132 INFO:teuthology.orchestra.run.smithi165.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:34:47.147 INFO:teuthology.orchestra.run.smithi165.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:34:47.149 INFO:teuthology.orchestra.run.smithi165.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:34:47.227 INFO:teuthology.orchestra.run.smithi165.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:34:47.422 INFO:teuthology.orchestra.run.smithi079.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:34:47.425 INFO:teuthology.orchestra.run.smithi079.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:34:47.479 INFO:teuthology.orchestra.run.smithi079.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:34:47.509 INFO:teuthology.orchestra.run.smithi165.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:34:47.510 INFO:teuthology.orchestra.run.smithi165.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:34:48.349 INFO:teuthology.orchestra.run.smithi118.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:34:48.349 INFO:teuthology.orchestra.run.smithi118.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:34:48.366 INFO:teuthology.orchestra.run.smithi118.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:34:48.462 INFO:teuthology.orchestra.run.smithi118.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:34:48.477 INFO:teuthology.orchestra.run.smithi118.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:34:48.518 INFO:teuthology.orchestra.run.smithi118.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:34:48.639 INFO:teuthology.orchestra.run.smithi118.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:34:49.047 INFO:teuthology.orchestra.run.smithi118.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:34:49.047 INFO:teuthology.orchestra.run.smithi118.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:34:49.397 INFO:teuthology.orchestra.run.smithi165.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:34:49.399 INFO:teuthology.orchestra.run.smithi165.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:34:49.400 INFO:teuthology.orchestra.run.smithi165.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:34:49.762 INFO:teuthology.orchestra.run.smithi165.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:34:49.763 INFO:teuthology.orchestra.run.smithi165.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:34:49.860 INFO:teuthology.orchestra.run.smithi079.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:34:50.024 INFO:teuthology.orchestra.run.smithi079.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:34:50.029 INFO:teuthology.orchestra.run.smithi079.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:34:50.032 INFO:teuthology.orchestra.run.smithi079.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:34:50.098 INFO:teuthology.orchestra.run.smithi079.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:34:50.335 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 181 MB in 5s (34.1 MB/s) 2024-09-21T23:34:50.373 INFO:teuthology.orchestra.run.smithi079.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:34:50.493 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:34:50.593 INFO:teuthology.orchestra.run.smithi165.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:34:50.600 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:34:50.683 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:34:50.903 INFO:teuthology.orchestra.run.smithi118.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:34:50.906 INFO:teuthology.orchestra.run.smithi118.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:34:50.906 INFO:teuthology.orchestra.run.smithi118.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:34:51.037 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:34:51.064 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:34:51.104 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:34:51.177 INFO:teuthology.orchestra.run.smithi118.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:34:51.180 INFO:teuthology.orchestra.run.smithi118.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:34:51.358 INFO:teuthology.orchestra.run.smithi079.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:34:51.365 INFO:teuthology.orchestra.run.smithi079.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:34:51.382 INFO:teuthology.orchestra.run.smithi079.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:34:51.424 INFO:teuthology.orchestra.run.smithi079.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:34:51.434 INFO:teuthology.orchestra.run.smithi079.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:34:51.436 INFO:teuthology.orchestra.run.smithi079.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:34:51.439 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:34:51.466 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:34:51.491 INFO:teuthology.orchestra.run.smithi079.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:34:51.506 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:34:51.725 INFO:teuthology.orchestra.run.smithi079.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:34:51.726 INFO:teuthology.orchestra.run.smithi079.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:34:51.751 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 181 MB in 7s (27.1 MB/s) 2024-09-21T23:34:51.916 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:34:51.934 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:34:51.961 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:52.009 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:52.017 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:34:52.022 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:34:52.109 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:34:52.428 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:34:52.454 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:34:52.455 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:52.480 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:34:52.504 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:52.521 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:34:52.824 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:34:52.850 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:34:52.856 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:34:52.875 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:52.890 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:34:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:53.284 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:34:53.310 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:34:53.336 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:34:53.348 INFO:teuthology.orchestra.run.smithi079.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:34:53.350 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:34:53.351 INFO:teuthology.orchestra.run.smithi079.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:34:53.352 INFO:teuthology.orchestra.run.smithi079.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:34:53.362 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:53.411 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:53.635 INFO:teuthology.orchestra.run.smithi079.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:34:53.637 INFO:teuthology.orchestra.run.smithi079.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:34:53.653 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librados2. 2024-09-21T23:34:53.672 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:53.711 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:53.756 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:34:53.782 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:53.814 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:54.098 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:34:54.114 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:54.151 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:34:54.156 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:54.177 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:54.197 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 181 MB in 9s (20.1 MB/s) 2024-09-21T23:34:54.217 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:54.341 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:34:54.435 INFO:teuthology.orchestra.run.smithi079.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:34:54.441 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:34:54.523 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:34:54.554 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:34:54.580 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:34:54.621 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:34:54.625 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:34:54.652 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:54.692 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:54.843 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:34:54.870 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:34:54.910 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:34:54.915 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librados2. 2024-09-21T23:34:54.942 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:54.982 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:55.194 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:34:55.220 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:55.229 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:34:55.256 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:34:55.260 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:55.296 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:34:55.435 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:34:55.461 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:55.510 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:55.563 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:34:55.578 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:55.613 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:55.708 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:34:55.735 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:55.783 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:55.815 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:34:55.831 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:55.865 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:55.990 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:34:56.014 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:56.055 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:56.176 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:34:56.178 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:34:56.202 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:56.204 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:56.243 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:56.244 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:56.341 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:34:56.368 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:56.408 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:34:56.579 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:34:56.599 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:34:56.605 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:34:56.631 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:34:56.646 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:34:56.703 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:34:56.728 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:56.769 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:56.948 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:34:56.975 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:34:57.015 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:34:57.059 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:34:57.086 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:34:57.125 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:34:57.234 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:34:57.249 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:57.264 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:34:57.284 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:57.290 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:57.330 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:57.453 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librados2. 2024-09-21T23:34:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:57.520 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:57.616 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:34:57.636 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:34:57.642 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:57.663 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:34:57.683 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:57.703 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:34:57.981 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:34:57.981 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:34:57.996 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:34:58.007 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:58.028 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:34:58.047 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:34:58.048 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:58.054 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:34:58.095 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:34:58.341 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:34:58.357 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:34:58.364 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:34:58.390 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:34:58.400 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:34:58.431 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:34:58.442 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:34:58.469 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:58.508 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:58.678 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:34:58.683 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:34:58.693 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:34:58.709 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:58.727 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:34:58.750 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:58.794 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:34:58.809 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:58.853 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:59.039 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:34:59.054 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:34:59.097 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:34:59.120 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:34:59.146 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:34:59.180 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:34:59.187 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:34:59.207 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:59.247 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:59.350 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package zip. 2024-09-21T23:34:59.376 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:34:59.416 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:34:59.448 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:34:59.463 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:34:59.498 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:34:59.499 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:34:59.526 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:34:59.566 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:34:59.711 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package unzip. 2024-09-21T23:34:59.737 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:34:59.777 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:34:59.784 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:34:59.799 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:34:59.835 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:34:59.851 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:34:59.862 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:34:59.902 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:00.125 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:35:00.149 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:35:00.189 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:35:00.196 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:35:00.222 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:35:00.255 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:35:00.263 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:35:00.282 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:35:00.322 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:35:00.574 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:35:00.589 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:35:00.632 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:35:00.666 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:35:00.693 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:35:00.733 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:35:00.868 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package zip. 2024-09-21T23:35:00.883 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:35:00.927 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:35:00.994 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:35:01.013 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:35:01.020 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:01.040 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:01.060 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:01.079 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:01.239 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package unzip. 2024-09-21T23:35:01.265 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:35:01.305 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:35:01.389 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:35:01.404 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:35:01.447 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:35:01.668 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:35:01.690 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:35:01.694 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:35:01.707 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:01.716 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:35:01.726 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:35:01.731 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:35:01.749 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:01.774 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:35:02.043 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:35:02.069 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:35:02.077 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:35:02.085 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:35:02.093 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:35:02.101 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:02.111 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:02.135 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:35:02.160 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:02.396 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:35:02.422 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:02.456 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:35:02.471 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:02.482 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:35:02.514 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:35:02.821 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:35:02.825 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:35:02.836 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:02.842 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:35:02.875 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:35:02.879 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:02.899 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:35:02.925 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:02.965 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:03.103 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package zip. 2024-09-21T23:35:03.129 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:35:03.169 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:35:03.191 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:35:03.217 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:35:03.257 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:03.481 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package unzip. 2024-09-21T23:35:03.507 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:35:03.544 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:35:03.547 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:35:03.570 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:03.611 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:03.934 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:35:03.960 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:35:04.001 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:35:04.031 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:35:04.057 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:04.097 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:04.312 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:35:04.338 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:04.387 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:04.409 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:35:04.435 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:04.475 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:04.916 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:35:04.931 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:35:04.966 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:35:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:05.285 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:35:05.311 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:35:05.351 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:35:05.386 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:05.678 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:35:05.696 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:35:05.704 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:05.705 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:35:05.723 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:35:05.732 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:35:05.745 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:05.762 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:05.772 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:06.058 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:35:06.065 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:35:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:06.091 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:35:06.125 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:06.132 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:35:06.387 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:35:06.401 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:35:06.413 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:35:06.427 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:35:06.457 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:06.459 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:35:06.620 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:35:06.646 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:06.686 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:06.745 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:35:06.918 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:35:06.918 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:35:06.918 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:35:06.918 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:35:06.918 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:35:07.114 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:35:07.130 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:35:07.139 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:35:07.165 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:35:07.173 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:35:07.205 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:07.459 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:35:07.485 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:35:07.525 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:35:07.625 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:35:07.652 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:35:07.683 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:35:07.753 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:35:07.768 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:35:07.803 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:35:07.961 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:35:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:35:08.028 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:35:08.069 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:35:08.095 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:08.145 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:08.148 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:35:08.174 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:35:08.214 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:35:08.331 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:35:08.358 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:35:08.398 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:35:08.525 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:35:08.552 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:35:08.592 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:35:08.673 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:35:08.676 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:35:08.700 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:35:08.702 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:35:08.732 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:08.742 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:35:08.886 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:35:08.913 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:35:08.944 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:35:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:35:09.045 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:35:09.072 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:35:09.085 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:35:09.112 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:35:09.222 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:35:09.248 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:35:09.289 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:35:09.398 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:35:09.405 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:35:09.425 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:35:09.431 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:35:09.465 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:35:09.471 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:09.642 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:35:09.668 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:35:09.708 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:35:09.783 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:35:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:35:09.809 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:35:09.821 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:35:09.849 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:35:09.860 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:35:10.011 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:35:10.038 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:35:10.078 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:35:10.086 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:35:10.112 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:35:10.152 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:35:10.172 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:35:10.199 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:35:10.239 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:35:10.363 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:35:10.390 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:35:10.430 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:35:10.447 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:35:10.474 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:35:10.514 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:35:10.542 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:35:10.568 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:35:10.608 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:10.741 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:35:10.756 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:35:10.793 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:10.825 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:35:10.852 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:35:10.853 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:35:10.868 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:35:10.892 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:35:10.903 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:35:11.044 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:35:11.070 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:35:11.110 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:35:11.179 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:35:11.198 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:35:11.205 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:35:11.225 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:35:11.245 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:35:11.265 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:35:11.421 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:35:11.448 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:35:11.488 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:35:11.506 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:35:11.533 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:35:11.568 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:35:11.573 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:35:11.594 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:35:11.643 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:35:11.774 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:35:11.789 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:35:11.832 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:35:11.910 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:35:11.925 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:35:11.954 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:35:11.968 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:35:11.981 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:35:12.021 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:35:12.118 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:35:12.145 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:35:12.184 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:35:12.305 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:35:12.332 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:35:12.372 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:35:12.374 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:35:12.401 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:35:12.435 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:12.512 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:35:12.539 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:35:12.596 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:35:12.658 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:35:12.674 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:35:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:12.720 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:35:12.747 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:35:12.786 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:35:12.907 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:35:12.934 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:35:12.973 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:35:13.011 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:35:13.038 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:35:13.078 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:35:13.097 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:35:13.124 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:35:13.164 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:35:13.319 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:35:13.346 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:35:13.385 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:35:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:35:13.441 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:35:13.450 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:35:13.457 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:35:13.490 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:35:13.500 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:35:13.691 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:35:13.718 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:35:13.757 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:35:13.776 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:35:13.803 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:35:13.803 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:35:13.829 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:35:13.843 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:35:13.870 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:35:14.076 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:35:14.103 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:35:14.121 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:35:14.143 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:35:14.147 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:35:14.187 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:35:14.223 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:35:14.249 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:35:14.289 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:35:14.413 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:35:14.439 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:35:14.470 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:35:14.507 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:35:14.533 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:35:14.585 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:14.609 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:35:14.635 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:35:14.676 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:35:14.757 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:35:14.784 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:35:14.823 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:35:14.919 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:35:14.945 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:35:14.985 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:35:15.073 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:35:15.099 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:35:15.139 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:35:15.143 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:35:15.158 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:15.201 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:15.289 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:35:15.315 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:35:15.355 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:35:15.476 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:35:15.502 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:35:15.564 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:35:15.567 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:35:15.591 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:15.630 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:15.686 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:35:15.710 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:35:15.750 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:35:15.854 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:35:15.880 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:35:15.912 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:35:15.984 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:35:16.010 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:35:16.050 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:16.053 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:35:16.080 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:35:16.112 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:35:16.190 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:35:16.217 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:35:16.249 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:35:16.407 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:35:16.433 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:35:16.473 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:35:16.561 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:35:16.585 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:35:16.588 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:35:16.612 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:35:16.619 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:35:16.652 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:35:16.785 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:35:16.811 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:35:16.851 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:35:16.939 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:35:16.965 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:17.005 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:17.442 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:35:17.468 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:17.508 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:17.748 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:35:17.774 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:17.819 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:18.023 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:35:18.050 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:35:18.090 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:35:18.337 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:35:18.362 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:18.402 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:35:18.402 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:18.428 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:35:18.460 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:35:18.639 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:35:18.665 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:35:18.705 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:18.755 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:35:18.781 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:35:18.821 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:35:18.871 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:35:18.897 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:18.936 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:19.033 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:35:19.060 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:35:19.100 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:35:19.141 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:35:19.168 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:35:19.208 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:35:19.420 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:35:19.447 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:19.487 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:19.511 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:35:19.537 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:35:19.578 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:35:19.889 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:35:19.890 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:35:19.915 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:19.917 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:19.956 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:19.956 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.031 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph. 2024-09-21T23:35:20.049 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:20.098 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.336 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:35:20.363 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:20.403 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.417 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:35:20.443 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:20.483 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.519 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:35:20.545 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:20.585 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.853 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:35:20.880 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:20.919 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:20.922 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:35:20.949 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:35:20.989 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:21.276 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:35:21.288 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:35:21.302 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:35:21.304 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:21.338 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:21.342 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:35:21.666 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:35:21.692 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:35:21.695 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:35:21.722 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:21.733 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:35:21.762 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:21.805 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph. 2024-09-21T23:35:21.822 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:21.863 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:22.043 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:35:22.059 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:22.093 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:22.132 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:35:22.159 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:22.199 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:22.208 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:35:22.235 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:22.275 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:22.604 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:35:22.630 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:22.670 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:22.878 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:35:22.904 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:22.944 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:23.090 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:35:23.098 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:35:23.117 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:23.125 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:23.156 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:23.164 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:23.451 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:35:23.478 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:35:23.509 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:35:23.509 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:35:23.536 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:35:23.576 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:35:23.896 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:35:23.911 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:23.937 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:35:23.953 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:23.963 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:24.003 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:24.064 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph. 2024-09-21T23:35:24.079 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:24.122 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:24.339 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:35:24.366 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:35:24.406 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:24.451 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:35:24.479 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:24.517 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:24.675 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:35:24.691 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:35:24.725 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:24.879 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:35:24.905 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:24.945 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:25.036 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:35:25.063 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:25.103 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:25.234 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:35:25.261 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:25.300 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:25.323 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:35:25.350 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:25.390 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:25.654 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:35:25.680 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:35:25.710 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:35:25.720 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:35:25.737 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:35:25.776 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:35:26.023 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:35:26.050 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:26.071 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:35:26.090 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:26.098 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:26.137 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:26.299 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:35:26.315 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:35:26.357 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:26.427 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:35:26.454 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:35:26.493 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:26.805 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:35:26.832 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:35:26.871 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:27.069 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:35:27.096 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:35:27.135 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:27.151 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:35:27.178 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:35:27.199 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:35:27.218 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:27.509 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:35:27.510 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:27.510 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:27.524 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:35:27.546 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:35:27.563 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:27.572 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:35:27.612 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:35:27.883 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:35:27.910 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:35:27.949 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:35:27.983 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:35:28.009 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:28.049 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:28.395 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:35:28.421 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:35:28.461 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:28.641 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:35:28.668 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:28.747 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:35:28.774 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:35:28.792 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:28.813 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:28.930 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:35:28.958 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:35:28.996 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:29.101 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:35:29.128 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:29.167 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:29.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:35:29.738 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:35:29.744 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:35:29.765 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:35:29.783 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:29.804 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:30.107 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:35:30.134 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:35:30.154 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:35:30.173 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:35:30.181 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:35:30.254 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:30.390 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:35:30.406 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:35:30.447 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:30.523 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:35:30.548 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:35:30.550 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:35:30.576 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:35:30.593 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:35:30.615 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:35:30.904 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:35:30.910 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:35:30.931 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:35:30.937 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:35:30.970 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:35:30.976 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:31.249 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:35:31.265 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:35:31.299 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:35:31.355 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:35:31.382 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:35:31.421 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:31.475 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:35:31.503 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:31.568 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:35:31.596 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:35:31.626 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:35:31.627 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:31.716 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:35:31.743 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:35:31.774 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:35:31.888 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:35:31.915 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:35:31.954 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:35:32.516 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:35:32.517 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:35:32.543 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:35:32.545 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:32.582 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:35:32.669 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:32.689 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:35:32.716 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:35:32.755 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:33.059 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:35:33.086 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:35:33.125 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:35:33.221 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:35:33.237 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:33.270 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:33.453 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:35:33.480 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:35:33.520 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:35:33.599 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:35:33.626 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:35:33.665 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:35:33.673 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:35:33.700 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:35:33.740 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:33.839 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:35:33.867 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:35:33.906 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:35:33.943 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:35:33.970 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:34.001 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:35:34.043 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:35:34.070 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:35:34.110 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:35:34.168 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:35:34.195 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:35:34.226 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:35:34.287 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package jq. 2024-09-21T23:35:34.315 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:34.353 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:35:34.430 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:35:34.458 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:35:34.496 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:35:34.529 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:35:34.556 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:35:34.580 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package socat. 2024-09-21T23:35:34.595 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:35:34.597 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:35:34.630 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:35:34.808 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:35:34.836 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:35:34.875 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:35:34.890 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:35:34.906 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:35:34.909 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:35:34.936 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:35:34.948 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:35:34.975 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:35:35.212 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:35:35.240 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:35:35.260 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:35:35.279 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:35:35.286 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:35:35.313 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:35:35.326 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:35:35.329 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:35.362 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:35.599 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:35:35.626 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:35:35.657 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:35:35.944 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:35:35.971 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:35:36.010 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:35:36.040 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:35:36.067 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:36.106 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:36.384 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:35:36.412 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:35:36.450 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:35:36.597 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:35:36.625 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:35:36.664 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:35:36.778 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:35:36.805 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:36.845 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:35:37.106 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package jq. 2024-09-21T23:35:37.133 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:37.172 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:35:37.320 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:35:37.348 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:37.386 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:37.425 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package socat. 2024-09-21T23:35:37.453 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:35:37.491 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:35:37.550 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:35:37.578 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:37.616 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:37.673 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:35:37.700 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:35:37.739 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:35:37.802 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:35:37.818 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:35:37.861 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:35:37.920 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:35:37.947 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:37.986 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:38.076 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:35:38.104 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:38.243 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:35:38.414 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:35:38.441 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:35:38.463 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:35:38.480 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:38.490 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:38.504 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package jq. 2024-09-21T23:35:38.532 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:35:38.557 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:38.570 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:35:38.790 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package socat. 2024-09-21T23:35:38.806 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:35:38.810 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:35:38.837 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:35:38.848 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:35:38.868 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:35:39.177 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:35:39.193 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:35:39.230 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:35:39.235 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:35:39.257 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:35:39.296 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:39.565 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:35:39.581 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:39.623 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:39.743 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:35:39.771 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:35:39.809 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:35:40.054 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:35:40.070 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:35:40.103 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:40.515 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:35:40.543 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:35:40.581 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:40.851 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:35:40.878 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:35:40.917 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:35:41.237 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:35:41.264 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:35:41.312 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:35:41.386 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:35:41.413 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:41.452 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:41.623 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:35:41.650 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:35:41.689 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:41.790 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:35:41.805 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:41.847 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:41.919 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:35:41.947 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:35:41.986 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:42.017 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:35:42.044 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:35:42.084 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:35:42.192 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:35:42.219 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:35:42.258 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:42.298 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:35:42.314 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:42.357 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:42.403 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:35:42.431 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:35:42.470 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:35:42.562 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:35:42.589 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:35:42.703 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:35:42.703 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:35:42.703 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:35:42.709 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:42.991 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:35:43.019 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:35:43.048 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:35:43.058 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:43.075 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:35:43.114 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:35:43.140 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:35:43.156 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:35:43.190 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:43.477 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:35:43.484 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:35:43.505 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:35:43.512 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:35:43.513 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:35:43.540 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:35:43.544 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:43.551 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:35:43.579 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:35:43.848 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:35:43.854 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:35:43.864 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:35:43.881 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:35:43.898 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:43.920 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:35:43.941 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:35:43.969 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:35:44.008 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:35:44.248 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:35:44.252 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:35:44.268 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:35:44.276 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:35:44.310 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:44.315 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:44.318 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:35:44.346 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:35:44.385 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:44.576 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:35:44.592 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:35:44.634 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:35:44.687 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:35:44.715 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:35:44.739 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:35:44.754 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:35:44.766 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:35:44.805 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:45.074 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:35:45.102 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:35:45.117 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:35:45.140 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:35:45.144 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:35:45.184 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:35:45.427 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:35:45.446 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:35:45.454 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:35:45.474 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:45.509 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:45.512 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:45.513 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:35:45.540 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:35:45.579 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:35:45.846 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:35:45.873 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:35:45.874 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:35:45.901 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:35:45.920 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:35:45.949 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:46.244 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:35:46.249 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:35:46.259 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:35:46.277 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:35:46.302 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:35:46.315 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:35:46.332 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:35:46.348 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:46.382 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:46.601 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:35:46.628 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:46.667 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:46.753 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:35:46.781 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:35:46.819 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:46.864 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:35:46.892 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:35:46.930 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:35:46.972 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:47.096 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:35:47.124 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:35:47.171 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:35:47.222 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:35:47.348 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:47.404 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:35:47.432 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:35:47.470 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:47.474 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:35:47.502 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:35:47.540 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:35:47.609 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:35:47.781 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:35:47.809 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:35:47.844 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:35:47.847 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:35:47.872 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:35:47.902 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:47.913 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:35:48.129 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:35:48.134 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:35:48.143 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:35:48.162 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:35:48.179 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:35:48.192 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:35:48.359 INFO:teuthology.orchestra.run.smithi165.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:35:48.446 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:48.496 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:35:48.524 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:35:48.563 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:48.815 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:35:48.831 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:35:48.874 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:35:49.007 INFO:teuthology.orchestra.run.smithi165.stdout:Adding system user cephadm....done 2024-09-21T23:35:49.163 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:49.175 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:35:49.203 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:49.241 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:49.448 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:49.670 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:35:49.682 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:49.698 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:49.736 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:49.808 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:50.061 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:35:50.076 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:50.088 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:50.127 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:50.352 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:50.372 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:35:50.400 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:50.438 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:50.453 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:35:50.531 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:35:50.547 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:50.589 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:50.726 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:50.738 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:50.851 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:35:50.878 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:35:50.909 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:50.959 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:35:51.086 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:51.090 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:51.197 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:35:51.349 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:35:51.366 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:35:51.439 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:35:51.566 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:35:51.642 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:35:51.668 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:35:51.826 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:35:51.877 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:52.107 INFO:teuthology.orchestra.run.smithi118.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:35:52.128 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:52.153 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:35:52.204 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:52.370 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:52.593 INFO:teuthology.orchestra.run.smithi079.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:35:52.622 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:52.676 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:52.762 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user cephadm....done 2024-09-21T23:35:52.881 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:52.918 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:52.982 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:53.100 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:35:53.175 INFO:teuthology.orchestra.run.smithi079.stdout:Adding system user cephadm....done 2024-09-21T23:35:53.203 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:53.225 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:53.291 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:35:53.463 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:53.476 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:35:53.567 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:35:53.597 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:53.832 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:53.835 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:35:53.862 INFO:teuthology.orchestra.run.smithi165.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:35:53.890 INFO:teuthology.orchestra.run.smithi165.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:35:53.946 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:35:53.961 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:35:54.063 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:54.083 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:54.213 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:35:54.217 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:35:54.415 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:54.489 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:35:54.535 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:54.541 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:35:54.615 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:35:54.817 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:54.917 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:35:54.930 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:55.084 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:55.188 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:35:55.277 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:35:55.385 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:35:55.481 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:35:55.511 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:35:55.537 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:35:55.732 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:55.797 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:35:55.967 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:56.082 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:35:56.180 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:35:56.218 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:56.341 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:35:56.433 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:35:56.559 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:35:56.569 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:56.601 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:35:56.685 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:35:56.852 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:56.903 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:56.969 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:35:57.029 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:57.163 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:35:57.178 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:35:57.289 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:57.313 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:35:57.346 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:35:57.439 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:35:57.549 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:35:57.565 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:35:57.588 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:35:57.824 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:35:57.964 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:35:58.020 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:35:58.021 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:35:58.060 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:35:58.186 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:58.305 INFO:teuthology.orchestra.run.smithi079.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:35:58.356 INFO:teuthology.orchestra.run.smithi079.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:35:58.420 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:35:58.450 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:35:58.546 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:35:58.553 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:58.679 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:35:58.818 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:35:58.930 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:35:58.981 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:59.056 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:35:59.061 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:35:59.162 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:35:59.240 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:59.324 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:35:59.496 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:35:59.533 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:35:59.575 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:35:59.642 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:35:59.806 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:35:59.885 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:36:00.011 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:36:00.302 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:36:00.578 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:36:00.688 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:36:00.704 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:36:00.839 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:36:00.928 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:36:00.939 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:36:01.066 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:36:01.131 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:36:01.172 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:01.200 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:36:01.281 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:36:01.493 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:36:01.508 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:36:01.532 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:36:01.659 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:36:01.750 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:36:01.784 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:36:01.878 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:36:02.061 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:36:02.154 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:36:02.168 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:36:02.320 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:36:02.453 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:36:02.571 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:02.673 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:36:02.812 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:36:02.855 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:36:02.966 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:36:03.072 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:36:03.148 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:36:03.173 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:03.217 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:36:03.468 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:36:03.482 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:36:03.500 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:36:03.702 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:36:03.751 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:36:03.817 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:36:03.878 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:03.970 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:36:04.171 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:36:04.230 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:36:04.331 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:36:04.480 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:36:04.598 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:36:04.725 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:36:05.009 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:36:05.014 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:36:05.257 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:05.293 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:36:05.375 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:36:05.528 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:36:05.654 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:36:05.659 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:36:05.661 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:36:05.780 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:36:05.785 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:36:05.869 INFO:teuthology.orchestra.run.smithi165.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:36:05.911 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:36:05.946 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:06.081 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:36:06.098 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:36:06.225 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:36:06.229 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:36:06.349 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:36:06.483 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:36:06.488 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:36:06.601 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:36:06.609 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:36:06.739 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:06.840 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:36:06.852 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:36:06.903 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:36:06.962 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:36:07.158 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:36:07.162 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:36:07.213 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:07.339 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:36:07.405 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:07.409 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:36:07.531 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:36:07.648 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:36:07.685 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:36:07.953 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:36:07.975 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:36:07.999 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:36:08.204 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:36:08.242 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:36:08.259 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:36:08.368 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:36:08.497 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:36:08.502 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:36:08.619 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:36:08.623 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:36:08.761 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:36:08.896 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:36:08.899 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:36:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:36:09.025 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:36:09.216 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:36:09.288 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:36:09.326 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:36:09.414 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:36:09.618 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:36:09.627 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:36:09.691 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:36:09.800 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:36:09.878 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:36:09.894 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:36:10.004 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:36:10.102 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:36:10.130 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:36:10.220 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:36:10.280 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:36:10.403 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:36:10.406 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:10.465 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:36:10.541 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:10.574 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:36:10.662 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:36:10.667 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:36:10.788 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:36:10.914 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:36:10.962 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:10.984 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:36:11.004 INFO:teuthology.orchestra.run.smithi079.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:36:11.087 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:11.214 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:36:11.243 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:36:11.250 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:36:11.369 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:36:11.376 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:36:11.524 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:11.653 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:11.667 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:11.778 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:36:11.779 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:36:11.793 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:11.997 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:11.997 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:12.037 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:36:12.156 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:36:12.163 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:36:12.431 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:36:12.465 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:36:12.510 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:12.557 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:36:12.645 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:12.763 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:12.774 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:36:12.867 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:36:12.889 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:36:12.917 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:36:13.168 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:36:13.193 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:36:13.456 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:36:13.477 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:36:13.587 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:36:13.831 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:36:13.854 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:36:14.094 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.122 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:36:14.204 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.224 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:36:14.329 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.456 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.485 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:36:14.492 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:36:14.581 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.716 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:14.870 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:36:14.886 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:36:15.012 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.138 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.163 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:36:15.208 INFO:teuthology.orchestra.run.smithi165.stdout:Adding group ceph....done 2024-09-21T23:36:15.264 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:36:15.500 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:36:15.562 INFO:teuthology.orchestra.run.smithi165.stdout:Adding system user ceph....done 2024-09-21T23:36:15.617 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.626 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.729 INFO:teuthology.orchestra.run.smithi165.stdout:Setting system user ceph properties....done 2024-09-21T23:36:15.734 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.747 INFO:teuthology.orchestra.run.smithi165.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:36:15.752 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:15.853 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:36:15.860 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:36:15.882 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:36:16.137 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.190 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.200 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:36:16.271 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.315 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.406 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.467 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:36:16.618 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:16.618 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:16.692 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.743 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.818 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.870 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:16.996 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.132 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.205 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:17.205 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:36:17.254 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:17.254 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:17.266 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.367 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.493 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:36:17.678 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.680 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.807 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:17.916 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:18.011 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:36:18.042 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:36:18.514 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:18.767 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:18.767 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:18.768 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:18.881 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:18.982 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.108 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.233 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.253 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.254 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.359 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.381 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.455 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:19.455 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:19.507 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.650 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.776 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.902 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:19.904 INFO:teuthology.orchestra.run.smithi118.stdout:Adding group ceph....done 2024-09-21T23:36:19.921 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:20.199 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:20.199 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:20.238 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user ceph....done 2024-09-21T23:36:20.373 INFO:teuthology.orchestra.run.smithi118.stdout:Setting system user ceph properties....done 2024-09-21T23:36:20.388 INFO:teuthology.orchestra.run.smithi118.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:36:20.454 INFO:teuthology.orchestra.run.smithi079.stdout:Adding group ceph....done 2024-09-21T23:36:20.522 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:36:20.714 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:20.837 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:36:20.839 INFO:teuthology.orchestra.run.smithi079.stdout:Adding system user ceph....done 2024-09-21T23:36:20.841 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:20.992 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:21.007 INFO:teuthology.orchestra.run.smithi079.stdout:Setting system user ceph properties....done 2024-09-21T23:36:21.024 INFO:teuthology.orchestra.run.smithi079.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:36:21.160 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:36:21.189 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:21.189 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:21.321 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:21.446 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:21.515 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:36:21.660 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:21.794 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:21.915 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:21.915 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:21.920 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:22.021 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:22.071 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:22.147 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:22.458 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:36:22.575 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:36:22.633 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:22.657 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:22.657 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:36:22.728 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:36:23.063 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:36:23.213 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:23.551 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:23.722 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:36:23.795 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:23.795 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:23.854 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:36:24.220 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:24.286 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:24.465 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:24.465 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:36:24.488 INFO:teuthology.orchestra.run.smithi165.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:36:24.492 DEBUG:teuthology.parallel:result is None 2024-09-21T23:36:24.503 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:24.503 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:24.979 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:25.182 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:25.183 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:36:25.338 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:25.338 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:25.657 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:25.892 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:25.942 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:25.943 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:36:26.025 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:26.176 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:26.390 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:26.390 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:26.425 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:26.526 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:26.661 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:26.873 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:26.873 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:36:27.036 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.279 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.320 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.405 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.463 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.547 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.589 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.741 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:36:27.884 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:36:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:36:28.094 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:36:28.145 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:36:28.194 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:36:28.482 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:36:29.200 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:36:29.237 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:36:29.890 INFO:teuthology.orchestra.run.smithi079.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:36:29.895 DEBUG:teuthology.parallel:result is None 2024-09-21T23:36:29.904 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:36:29.909 DEBUG:teuthology.parallel:result is None 2024-09-21T23:36:29.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:36:30.058 DEBUG:teuthology.orchestra.run.smithi079:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:36:30.077 INFO:teuthology.orchestra.run.smithi079.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.077 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.077 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:36:30.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:36:30.224 DEBUG:teuthology.orchestra.run.smithi118:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:36:30.242 INFO:teuthology.orchestra.run.smithi118.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.243 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.243 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:36:30.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:36:30.393 DEBUG:teuthology.orchestra.run.smithi165:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:36:30.413 INFO:teuthology.orchestra.run.smithi165.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.413 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:36:30.413 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:36:30.415 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T23:36:30.415 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:30.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:36:30.430 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:30.430 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:36:30.445 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:30.445 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:36:30.469 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T23:36:30.469 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:30.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:36:30.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:36:30.538 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:30.538 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:36:30.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:36:30.607 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:30.607 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:36:30.621 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:36:30.677 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T23:36:30.677 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:30.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:36:30.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:36:30.746 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:30.746 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:36:30.759 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:36:30.814 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:30.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:36:30.827 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:36:30.882 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T23:36:30.883 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:30.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:36:30.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:36:30.951 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:30.951 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:36:30.964 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:36:31.018 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:31.018 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:36:31.032 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:36:31.087 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T23:36:31.164 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T23:36:31.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2024-09-21T23:36:31.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /var/log/ceph 2024-09-21T23:36:31.170 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-09-21T23:36:31.183 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T23:36:31.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:36:31.228 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:36:31.231 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:36:31.243 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T23:36:31.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:36:31.284 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:36:31.287 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:36:31.308 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T23:36:31.308 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T23:36:31.308 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914530', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGgeet36R+jfq/IFcfuETTNF5qTP7zALK6Dmvx1qL28s51kYmxcjdQ01FrlhzLucpHynO1QWGZE3XTjMZfYb/D0=', 'smithi118.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBINzdY72cKL+DRCr5QLPb/7wgaDGwPqF2x3KuBPR9yJ8MMoARZcFR0bR+Y+X+9jDoi+KSoMEPN7K+UXDFz4PCKs=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJhYYT2yW1ohXK2PwQ58q8HJ8+z2zNphvoxX5w4M5Ax0J96jzVqJXDXkZuLNbdCKYd6wRruhnFIM/fDIYdIGjhA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'workunit': {'clients': {'client.0': ['rbd/rbd-nbd.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T23:36:31.308 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:36:31.340 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:36:31.343 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:36:31.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:36:31.388 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:36:31.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:36:31.405 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:31.405 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:36:31.447 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:36:31.447 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-09-21T23:36:31.495 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:36:31.495 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 23:31:02.902392962 +0000 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 23:31:02.662395106 +0000 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 23:31:02.662395106 +0000 2024-09-21T23:36:31.496 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-21T23:36:31.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:36:31.550 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T23:36:31.550 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T23:36:31.550 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000451493 s, 1.1 MB/s 2024-09-21T23:36:31.552 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:36:31.601 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 23:31:03.378388710 +0000 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 23:31:03.122390997 +0000 2024-09-21T23:36:31.647 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 23:31:03.122390997 +0000 2024-09-21T23:36:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-21T23:36:31.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T23:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T23:36:31.701 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00041039 s, 1.2 MB/s 2024-09-21T23:36:31.703 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:36:31.753 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 23:31:03.634386423 +0000 2024-09-21T23:36:31.799 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 23:31:03.630386459 +0000 2024-09-21T23:36:31.800 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 23:31:03.630386459 +0000 2024-09-21T23:36:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-21T23:36:31.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:36:31.853 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T23:36:31.853 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T23:36:31.853 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000395618 s, 1.3 MB/s 2024-09-21T23:36:31.855 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:36:31.904 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-09-21T23:36:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:36:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:31.955 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-21T23:36:31.955 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:31.955 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-21 23:31:04.374379814 +0000 2024-09-21T23:36:31.956 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-21 23:31:04.126382029 +0000 2024-09-21T23:36:31.956 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-21 23:31:04.126382029 +0000 2024-09-21T23:36:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-21T23:36:31.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:36:32.010 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-21T23:36:32.010 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-21T23:36:32.010 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00039123 s, 1.3 MB/s 2024-09-21T23:36:32.011 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:36:32.061 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:32.061 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:32.061 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:36:32.067 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:36:32.067 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-21 23:31:00.298692865 +0000 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-21 23:31:00.034695223 +0000 2024-09-21T23:36:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-21 23:31:00.034695223 +0000 2024-09-21T23:36:32.117 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-21T23:36:32.117 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:36:32.170 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-21T23:36:32.170 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-21T23:36:32.170 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000315664 s, 1.6 MB/s 2024-09-21T23:36:32.171 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:36:32.222 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-09-21T23:36:32.271 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:36:32.271 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:32.271 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-21T23:36:32.271 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:32.272 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-21 23:31:00.774688613 +0000 2024-09-21T23:36:32.272 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-21 23:31:00.522690864 +0000 2024-09-21T23:36:32.272 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-21 23:31:00.522690864 +0000 2024-09-21T23:36:32.272 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-21T23:36:32.272 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:36:32.325 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-21T23:36:32.326 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-21T23:36:32.326 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000261628 s, 2.0 MB/s 2024-09-21T23:36:32.327 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:36:32.377 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-21 23:31:01.238684468 +0000 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-21 23:31:00.986686720 +0000 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-21 23:31:00.986686720 +0000 2024-09-21T23:36:32.424 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-21T23:36:32.424 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:36:32.478 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-21T23:36:32.478 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-21T23:36:32.478 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000319446 s, 1.6 MB/s 2024-09-21T23:36:32.479 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:36:32.529 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-09-21T23:36:32.575 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:36:32.575 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:36:32.575 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-09-21T23:36:32.575 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:36:32.576 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-21 23:31:01.466682432 +0000 2024-09-21T23:36:32.576 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-21 23:31:01.462682468 +0000 2024-09-21T23:36:32.576 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-21 23:31:01.462682468 +0000 2024-09-21T23:36:32.576 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-21T23:36:32.576 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:36:32.630 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-21T23:36:32.630 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-21T23:36:32.630 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000307438 s, 1.7 MB/s 2024-09-21T23:36:32.631 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:36:32.681 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:32.681 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T23:36:32.681 INFO:tasks.ceph:Generating config... 2024-09-21T23:36:32.682 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T23:36:32.682 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T23:36:32.682 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T23:36:32.682 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T23:36:32.682 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T23:36:32.683 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T23:36:32.683 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T23:36:32.683 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T23:36:32.683 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T23:36:32.684 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T23:36:32.684 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T23:36:32.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T23:36:32.719 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T23:36:32.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T23:36:32.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:36:32.867 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.c', '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]'), ('mon.b', '172.21.15.118')] 2024-09-21T23:36:32.867 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.118', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T23:36:32.868 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:32.868 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:36:32.915 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --addv c '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]' --add b 172.21.15.118 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:36:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = quincy 2024-09-21T23:36:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2024-09-21T23:36:33.001370+0000 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-09-21T23:36:33.001370+0000 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 17 (quincy) 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 1 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.b 2024-09-21T23:36:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] mon.c 2024-09-21T23:36:33.003 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T23:36:33.004 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:36:33.052 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 66973394-6d88-4f14-bf11-7ccd7c9cf4a7... 2024-09-21T23:36:33.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:36:33.096 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:36:33.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2024-09-21T23:36:33.126 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:36:33.127 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2024-09-21T23:36:33.128 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.118" 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: mon client directed command retry = 5 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on pool no app = False 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: ms inject socket failures = 5000 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:36:33.129 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug trim objects = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.130 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bdev async discard = True 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bdev enable discard = True 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore allocator = stupid 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2024-09-21T23:36:33.131 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 1/20 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 1/20 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 4/10 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: osd max markdown count = 1000 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-21T23:36:33.132 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2024-09-21T23:36:33.133 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:36:33.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2024-09-21T23:36:33.135 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-09-21T23:36:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.118" 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: mon client directed command retry = 5 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on pool no app = False 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: ms inject socket failures = 5000 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-09-21T23:36:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = stupid 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-09-21T23:36:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 1/20 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 1/20 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 4/10 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: osd max markdown count = 1000 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-09-21T23:36:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-09-21T23:36:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:36:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-09-21T23:36:33.145 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2024-09-21T23:36:33.146 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:36:33.147 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon_allow_pool_size_one = true 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon health detail to clog = false 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.118" 2024-09-21T23:36:33.148 INFO:teuthology.orchestra.run.smithi118.stdout: mon client directed command retry = 5 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on pool no app = False 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: ms inject socket failures = 5000 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.149 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug trim objects = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2024-09-21T23:36:33.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bdev async discard = True 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bdev enable discard = True 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore allocator = stupid 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 1/20 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 1/20 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-21T23:36:33.151 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 20 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 4/10 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: osd max markdown count = 1000 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-09-21T23:36:33.152 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.153 INFO:teuthology.orchestra.run.smithi118.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: # 1m isn't quite enough 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2024-09-21T23:36:33.154 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2024-09-21T23:36:33.155 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:36:33.155 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:36:33.155 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2024-09-21T23:36:33.155 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2024-09-21T23:36:33.155 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2024-09-21T23:36:33.156 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T23:36:33.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T23:36:33.247 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T23:36:33.248 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:33.248 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T23:36:33.253 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:33.253 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T23:36:33.300 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:36:33.300 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:33.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:36:33.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:36:33.364 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:33.364 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:33.413 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:36:33.413 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:33.413 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:36:33.413 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:36:33.435 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:33.435 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:33.488 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-09-21T23:36:33.488 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:33.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:36:33.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:36:33.511 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:33.511 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:33.560 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T23:36:33.560 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T23:36:33.560 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:36:33.603 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:36:33.618 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T23:36:33.618 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T23:36:33.618 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T23:36:33.653 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T23:36:33.675 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T23:36:33.675 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T23:36:33.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T23:36:33.689 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:33.689 INFO:tasks.ceph:role: osd.0 2024-09-21T23:36:33.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:36:33.690 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:36:33.751 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:34.212 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-21T23:36:34.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-21T23:36:34.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T23:36:34.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T23:36:34.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:34.322 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:34.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T23:36:34.374 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:34.374 INFO:tasks.ceph:role: osd.1 2024-09-21T23:36:34.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:36:34.375 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:36:34.438 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:34.439 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:34.906 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-21T23:36:34.909 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-21T23:36:34.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T23:36:35.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T23:36:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:35.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:35.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T23:36:35.122 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:35.122 INFO:tasks.ceph:role: osd.2 2024-09-21T23:36:35.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:36:35.123 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:35.187 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:35.666 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-21T23:36:35.668 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-21T23:36:35.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T23:36:35.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T23:36:35.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:35.811 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:35.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:35.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:35.914+0000 7fdecef827c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T23:36:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:35.914+0000 7fdecef827c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T23:36:35.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:35.914+0000 7fdecef827c0 -1 bdev(0x5562b74cf800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:35.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:35.914+0000 7fdecef827c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T23:36:39.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T23:36:39.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:39.162+0000 7f0a061e77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T23:36:39.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:39.162+0000 7f0a061e77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T23:36:39.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:39.162+0000 7f0a061e77c0 -1 bdev(0x558377fa3800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:39.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:39.162+0000 7f0a061e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T23:36:42.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T23:36:42.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:42.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:42.454+0000 7f3a34f707c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T23:36:42.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:42.454+0000 7f3a34f707c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T23:36:42.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:42.454+0000 7f3a34f707c0 -1 bdev(0x558ef4cd7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:42.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:36:42.454+0000 7f3a34f707c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T23:36:45.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T23:36:45.655 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T23:36:45.670 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:45.670 INFO:tasks.ceph:role: osd.3 2024-09-21T23:36:45.670 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:36:45.670 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:36:45.723 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:45.724 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:45.747 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2024-09-21T23:36:45.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-21T23:36:45.749 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-21T23:36:45.807 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T23:36:45.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:45.862 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:45.862 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T23:36:45.914 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:45.914 INFO:tasks.ceph:role: osd.4 2024-09-21T23:36:45.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:36:45.914 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:36:45.967 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:45.968 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:45.991 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2024-09-21T23:36:45.993 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-21T23:36:45.993 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-21T23:36:46.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T23:36:46.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:46.105 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:46.106 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T23:36:46.158 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:36:46.158 INFO:tasks.ceph:role: osd.5 2024-09-21T23:36:46.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:36:46.158 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:36:46.210 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:36:46.234 INFO:teuthology.orchestra.run.smithi118.stdout:Discarding blocks...Done. 2024-09-21T23:36:46.235 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-21T23:36:46.236 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-21T23:36:46.294 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T23:36:46.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:36:46.348 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:36:46.348 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:46.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:46.459+0000 7f95281797c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T23:36:46.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:46.459+0000 7f95281797c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T23:36:46.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:46.459+0000 7f95281797c0 -1 bdev(0x556e7883d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:46.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:46.459+0000 7f95281797c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T23:36:49.628 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T23:36:49.641 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:49.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:49.719+0000 7f9c571d07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T23:36:49.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:49.719+0000 7f9c571d07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T23:36:49.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:49.719+0000 7f9c571d07c0 -1 bdev(0x56135d53b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:49.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:49.719+0000 7f9c571d07c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T23:36:52.874 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T23:36:52.936 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:53.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:53.015+0000 7f2bd801a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T23:36:53.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:53.015+0000 7f2bd801a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T23:36:53.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:53.015+0000 7f2bd801a7c0 -1 bdev(0x557b1ebe1800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T23:36:53.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-21T23:36:53.015+0000 7f2bd801a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T23:36:56.178 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T23:36:56.231 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T23:36:56.231 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:56.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T23:36:56.245 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:56.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T23:36:56.298 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:56.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T23:36:56.354 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:56.354 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T23:36:56.368 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:56.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T23:36:56.422 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:56.422 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T23:36:56.474 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:56.474 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T23:36:56.530 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:36:56.530 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T23:36:56.536 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T23:36:56.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:36:56.538 DEBUG:teuthology.orchestra.run.smithi118:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCDWO9mgVOVNhAADkCdJi8noXrYOHQRlMrOHA== 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCHWO9m/LngCRAA5BmTV7UPy+szNVskS3Pijw== 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCKWO9mZfNNGxAAhNDOTy3S3ir+Yb1CmK+jiQ== 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2024-09-21T23:36:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCBWO9mnjw5JBAA81u+2GAOdtHRgv4VjSIQ4A== 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCOWO9mzFCnGxAASVReJZii0SYXHdoJTEYHCw== 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCRWO9mOa88KxAAP5jfmP7cfXkbZmFMFJEVRw== 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCVWO9mn8FJARAAmsclGGo8nibTRK4aGm6mUQ== 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-09-21T23:36:56.574 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCBWO9mC3YwJxAAzxdME74W6NpHQUmi8FpW7g== 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.0] 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCDWO9mgVOVNhAADkCdJi8noXrYOHQRlMrOHA== 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.1] 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCHWO9m/LngCRAA5BmTV7UPy+szNVskS3Pijw== 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.2] 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCKWO9mZfNNGxAAhNDOTy3S3ir+Yb1CmK+jiQ== 2024-09-21T23:36:56.580 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.x] 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCBWO9mnjw5JBAA81u+2GAOdtHRgv4VjSIQ4A== 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.3] 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCOWO9mzFCnGxAASVReJZii0SYXHdoJTEYHCw== 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.4] 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCRWO9mOa88KxAAP5jfmP7cfXkbZmFMFJEVRw== 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.5] 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCVWO9mn8FJARAAmsclGGo8nibTRK4aGm6mUQ== 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2024-09-21T23:36:56.581 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQCBWO9mC3YwJxAAzxdME74W6NpHQUmi8FpW7g== 2024-09-21T23:36:56.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.624 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.753 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:36:56.836 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:36:56.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:56.996 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:57.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:57.080 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:36:57.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:36:57.164 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:36:57.237 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T23:36:57.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T23:36:57.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:36:57.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T23:36:57.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T23:36:57.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:36:57.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T23:36:57.565 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T23:36:57.579 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:36:57.674 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T23:36:57.684 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:57.687 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:36:57.731 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T23:36:57.731 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T23:36:57.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T23:36:57.733 INFO:tasks.ceph.mon.a:Started 2024-09-21T23:36:57.734 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T23:36:57.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T23:36:57.736 INFO:tasks.ceph.mon.c:Started 2024-09-21T23:36:57.736 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T23:36:57.736 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T23:36:57.776 INFO:tasks.ceph.mon.b:Started 2024-09-21T23:36:57.776 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T23:36:57.776 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T23:36:57.776 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T23:36:57.779 INFO:tasks.ceph.mgr.x:Started 2024-09-21T23:36:57.779 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T23:36:57.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T23:36:58.203 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2024-09-21T23:36:58.222 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T23:36:58.222 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T23:36:58.223 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:58.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T23:36:58.234 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:58.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T23:36:58.288 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-21T23:36:58.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T23:36:58.306 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:58.303+0000 7f96482d4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T23:36:58.342 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:58.342 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T23:36:58.356 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:58.357 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T23:36:58.411 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-21T23:36:58.411 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T23:36:58.467 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 638e3e48-1763-4ec9-8c06-634e0b3b793b 0 2024-09-21T23:36:58.715 INFO:teuthology.orchestra.run.smithi118.stdout:0 2024-09-21T23:36:58.732 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 02967526-d5c5-4f68-8e25-343311dbaec4 1 2024-09-21T23:36:58.776 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:58.775+0000 7f96482d4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T23:36:58.927 INFO:teuthology.orchestra.run.smithi118.stdout:1 2024-09-21T23:36:58.943 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 4f770ac1-d062-4934-81d5-7f980b3830a1 2 2024-09-21T23:36:58.985 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:58.983+0000 7f96482d4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T23:36:59.184 INFO:teuthology.orchestra.run.smithi118.stdout:2 2024-09-21T23:36:59.199 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 90cc3313-3845-48d0-839d-8fea4c52bfde 3 2024-09-21T23:36:59.398 INFO:teuthology.orchestra.run.smithi118.stdout:3 2024-09-21T23:36:59.414 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 42b8e641-af88-4d41-bc5f-a14020f205ce 4 2024-09-21T23:36:59.573 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.571+0000 7f96482d4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T23:36:59.616 INFO:teuthology.orchestra.run.smithi118.stdout:4 2024-09-21T23:36:59.635 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.635+0000 7f96482d4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T23:36:59.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd new 2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9 5 2024-09-21T23:36:59.690 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.687+0000 7f96482d4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T23:36:59.770 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.767+0000 7f96482d4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T23:36:59.822 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.819+0000 7f96482d4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T23:36:59.829 INFO:teuthology.orchestra.run.smithi118.stdout:5 2024-09-21T23:36:59.845 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T23:36:59.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T23:36:59.848 INFO:tasks.ceph.osd.0:Started 2024-09-21T23:36:59.848 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T23:36:59.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T23:36:59.850 INFO:tasks.ceph.osd.1:Started 2024-09-21T23:36:59.851 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T23:36:59.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T23:36:59.853 INFO:tasks.ceph.osd.2:Started 2024-09-21T23:36:59.853 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T23:36:59.853 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T23:36:59.855 INFO:tasks.ceph.osd.3:Started 2024-09-21T23:36:59.855 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T23:36:59.855 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T23:36:59.857 INFO:tasks.ceph.osd.4:Started 2024-09-21T23:36:59.857 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T23:36:59.857 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T23:36:59.858 INFO:tasks.ceph.osd.5:Started 2024-09-21T23:36:59.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:36:59.907 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.907+0000 7f96482d4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T23:36:59.963 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:36:59.963+0000 7f96482d4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T23:37:00.045 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":8,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:36:59.825771+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:00.062 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T23:37:00.062 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T23:37:00.115 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:00.115+0000 7f96482d4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T23:37:00.176 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:00.175+0000 7f96482d4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T23:37:00.666 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:37:00.883 INFO:tasks.ceph.mgr.x.smithi118.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T23:37:00.883 INFO:tasks.ceph.mgr.x.smithi118.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T23:37:00.883 INFO:tasks.ceph.mgr.x.smithi118.stderr: from numpy import show_config as show_numpy_config 2024-09-21T23:37:00.950 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:00.947+0000 7f96482d4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T23:37:00.991 INFO:teuthology.misc.health.smithi079.stderr:2024-09-21T23:37:00.990+0000 7faa16ffd640 0 --2- 172.21.15.79:0/1048303751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faa1809ac80 0x7faa1809efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T23:37:01.079 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-21T23:37:01.079 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":8,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:36:59.825771+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:01.096 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-21T23:37:01.128 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:01.127+0000 7f96482d4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:37:01.224 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:01.223+0000 7f2cf96eb7c0 -1 Falling back to public interface 2024-09-21T23:37:01.418 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:01.415+0000 7f96482d4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T23:37:01.472 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:01.470+0000 7fbedb6bb7c0 -1 Falling back to public interface 2024-09-21T23:37:01.485 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:01.483+0000 7f96482d4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T23:37:01.495 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:01.495+0000 7fa2063627c0 -1 Falling back to public interface 2024-09-21T23:37:01.536 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:01.534+0000 7fa4e37fb7c0 -1 Falling back to public interface 2024-09-21T23:37:01.555 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:01.554+0000 7fb5b85817c0 -1 Falling back to public interface 2024-09-21T23:37:01.621 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:01.619+0000 7f96482d4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:37:01.984 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:01.983+0000 7ff1ce07d7c0 -1 Falling back to public interface 2024-09-21T23:37:02.230 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.231+0000 7f96482d4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T23:37:02.305 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.303+0000 7f96482d4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T23:37:02.488 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.487+0000 7f96482d4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T23:37:02.561 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.559+0000 7f96482d4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T23:37:02.852 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.851+0000 7f96482d4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T23:37:02.914 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:02.911+0000 7f96482d4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T23:37:03.951 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:03.951+0000 7f95eee08640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T23:37:03.952 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:37:03.951+0000 7f95eee08640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T23:37:05.708 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:05.706+0000 7fa4e37fb7c0 -1 osd.1 0 log_to_monitors true 2024-09-21T23:37:05.789 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:05.787+0000 7fa2063627c0 -1 osd.3 0 log_to_monitors true 2024-09-21T23:37:05.830 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:05.830+0000 7fbedb6bb7c0 -1 osd.0 0 log_to_monitors true 2024-09-21T23:37:05.967 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:05.967+0000 7f2cf96eb7c0 -1 osd.4 0 log_to_monitors true 2024-09-21T23:37:05.992 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:05.990+0000 7fb5b85817c0 -1 osd.2 0 log_to_monitors true 2024-09-21T23:37:06.038 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:06.035+0000 7ff1ce07d7c0 -1 osd.5 0 log_to_monitors true 2024-09-21T23:37:07.703 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:37:08.038 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-21T23:37:08.038 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:37:07.006461+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:08.057 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-21T23:37:08.158 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:08.155+0000 7fa202b1d640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T23:37:08.175 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:08.175+0000 7f2cf4e71640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T23:37:08.183 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:08.183+0000 7fa1fa12f640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:08.208 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:08.207+0000 7f2ced4b8640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:08.928 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:08.927+0000 7ff1c9803640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T23:37:08.967 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:08.967+0000 7ff1c1e4a640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:08.989 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:08.986+0000 7fb5b3d07640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T23:37:08.992 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:08.990+0000 7fa4def81640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T23:37:09.015 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:09.014+0000 7fb5ac34e640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:09.027 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:09.026+0000 7fa4d75c8640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:09.032 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:09.030+0000 7fbed7e76640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T23:37:09.067 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:09.066+0000 7fbecf488640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:37:14.662 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:37:14.961 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-21T23:37:14.962 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":16,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:37:13.216883+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:14.979 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-21T23:37:14.979 INFO:tasks.ceph:Creating RBD pool 2024-09-21T23:37:14.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T23:37:16.249 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'rbd' created 2024-09-21T23:37:16.274 DEBUG:teuthology.orchestra.run.smithi079:> rbd --cluster ceph pool init rbd 2024-09-21T23:37:19.287 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T23:37:19.287 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T23:37:19.288 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T23:37:19.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T23:37:19.288 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T23:37:19.636 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:19.653 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3383731644},{"type":"v1","addr":"172.21.15.118:6825","nonce":3383731644}]},"active_addr":"172.21.15.118:6825/3383731644","active_change":"2024-09-21T23:37:02.934589+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2073123306}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":982308143}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":1949114749}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":4116426360}]}]} 2024-09-21T23:37:19.655 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T23:37:19.655 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:37:19.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:37:19.948 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:20.087 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:37:19.261056+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:37:15.320283+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:20.088 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:37:20.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:37:20.394 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:20.395 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":20,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:37:19.261056+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:37:15.320283+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:37:20.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:37:20.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:37:20.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:37:20.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:37:20.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:37:20.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:37:20.585 INFO:teuthology.orchestra.run.smithi079.stdout:51539607559 2024-09-21T23:37:20.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:37:20.602 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:20.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:37:20.637 INFO:teuthology.orchestra.run.smithi079.stdout:51539607557 2024-09-21T23:37:20.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:37:20.655 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:20.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:37:20.663 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:20.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:37:20.682 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:20.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:37:20.930 INFO:teuthology.orchestra.run.smithi079.stdout:51539607557 2024-09-21T23:37:20.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607559 got 51539607557 for osd.4 2024-09-21T23:37:21.009 INFO:teuthology.orchestra.run.smithi079.stdout:51539607556 2024-09-21T23:37:21.013 INFO:teuthology.orchestra.run.smithi079.stdout:51539607556 2024-09-21T23:37:21.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.3 2024-09-21T23:37:21.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.2 2024-09-21T23:37:21.048 INFO:teuthology.orchestra.run.smithi079.stdout:51539607555 2024-09-21T23:37:21.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.1 2024-09-21T23:37:21.089 INFO:teuthology.orchestra.run.smithi079.stdout:51539607556 2024-09-21T23:37:21.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.0 2024-09-21T23:37:21.156 INFO:teuthology.orchestra.run.smithi079.stdout:51539607556 2024-09-21T23:37:21.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.5 2024-09-21T23:37:21.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:37:22.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:37:22.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:37:22.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:37:22.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:37:22.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:37:22.286 INFO:teuthology.orchestra.run.smithi079.stdout:51539607559 2024-09-21T23:37:22.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607559 got 51539607559 for osd.4 2024-09-21T23:37:22.305 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.394 INFO:teuthology.orchestra.run.smithi079.stdout:51539607557 2024-09-21T23:37:22.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.1 2024-09-21T23:37:22.411 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.412 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:22.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.3 2024-09-21T23:37:22.429 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.478 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:22.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.2 2024-09-21T23:37:22.495 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.508 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:22.520 INFO:teuthology.orchestra.run.smithi079.stdout:51539607558 2024-09-21T23:37:22.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.5 2024-09-21T23:37:22.526 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.0 2024-09-21T23:37:22.537 DEBUG:teuthology.parallel:result is None 2024-09-21T23:37:22.537 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:37:22.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:22.854 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:22.870 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T23:37:22.828504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":189440,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566041600,"statfs":{"total":579820584960,"available":579626598400,"internally_reserved":0,"allocated":2326528,"data_stored":1561900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.877575"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277603+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:19.277603+0000","last_peered":"2024-09-21T23:37:19.277603+0000","last_clean":"2024-09-21T23:37:19.277603+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:19.277603+0000","last_undegraded":"2024-09-21T23:37:19.277603+0000","last_fullsized":"2024-09-21T23:37:19.277603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272836+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:19.272836+0000","last_peered":"2024-09-21T23:37:19.272836+0000","last_clean":"2024-09-21T23:37:19.272836+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:19.272836+0000","last_undegraded":"2024-09-21T23:37:19.272836+0000","last_fullsized":"2024-09-21T23:37:19.272836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277632+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:19.277632+0000","last_peered":"2024-09-21T23:37:19.277632+0000","last_clean":"2024-09-21T23:37:19.277632+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:19.277632+0000","last_undegraded":"2024-09-21T23:37:19.277632+0000","last_fullsized":"2024-09-21T23:37:19.277632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272996+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:19.272996+0000","last_peered":"2024-09-21T23:37:19.272996+0000","last_clean":"2024-09-21T23:37:19.272996+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:19.272996+0000","last_undegraded":"2024-09-21T23:37:19.272996+0000","last_fullsized":"2024-09-21T23:37:19.272996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.285382+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:19.285382+0000","last_peered":"2024-09-21T23:37:19.285382+0000","last_clean":"2024-09-21T23:37:19.285382+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:19.285382+0000","last_undegraded":"2024-09-21T23:37:19.285382+0000","last_fullsized":"2024-09-21T23:37:19.285382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.296725+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:19.296725+0000","last_peered":"2024-09-21T23:37:19.296725+0000","last_clean":"2024-09-21T23:37:19.296725+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:19.296725+0000","last_undegraded":"2024-09-21T23:37:19.296725+0000","last_fullsized":"2024-09-21T23:37:19.296725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.278536+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:19.278536+0000","last_peered":"2024-09-21T23:37:19.278536+0000","last_clean":"2024-09-21T23:37:19.278536+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:19.278536+0000","last_undegraded":"2024-09-21T23:37:19.278536+0000","last_fullsized":"2024-09-21T23:37:19.278536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.282766+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:19.282766+0000","last_peered":"2024-09-21T23:37:19.282766+0000","last_clean":"2024-09-21T23:37:19.282766+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:19.282766+0000","last_undegraded":"2024-09-21T23:37:19.282766+0000","last_fullsized":"2024-09-21T23:37:19.282766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.276165+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:19.276165+0000","last_peered":"2024-09-21T23:37:19.276165+0000","last_clean":"2024-09-21T23:37:19.276165+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:19.276165+0000","last_undegraded":"2024-09-21T23:37:19.276165+0000","last_fullsized":"2024-09-21T23:37:19.276165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336300,"statfs":{"total":96636764160,"available":96600371200,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:22.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:23.144 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:23.161 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T23:37:22.952543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":189440,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566041600,"statfs":{"total":579820584960,"available":579626598400,"internally_reserved":0,"allocated":2326528,"data_stored":1561900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.694850"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277603+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:19.277603+0000","last_peered":"2024-09-21T23:37:19.277603+0000","last_clean":"2024-09-21T23:37:19.277603+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:19.277603+0000","last_undegraded":"2024-09-21T23:37:19.277603+0000","last_fullsized":"2024-09-21T23:37:19.277603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272836+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:19.272836+0000","last_peered":"2024-09-21T23:37:19.272836+0000","last_clean":"2024-09-21T23:37:19.272836+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:19.272836+0000","last_undegraded":"2024-09-21T23:37:19.272836+0000","last_fullsized":"2024-09-21T23:37:19.272836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277632+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:19.277632+0000","last_peered":"2024-09-21T23:37:19.277632+0000","last_clean":"2024-09-21T23:37:19.277632+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:19.277632+0000","last_undegraded":"2024-09-21T23:37:19.277632+0000","last_fullsized":"2024-09-21T23:37:19.277632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272996+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:19.272996+0000","last_peered":"2024-09-21T23:37:19.272996+0000","last_clean":"2024-09-21T23:37:19.272996+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:19.272996+0000","last_undegraded":"2024-09-21T23:37:19.272996+0000","last_fullsized":"2024-09-21T23:37:19.272996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.285382+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:19.285382+0000","last_peered":"2024-09-21T23:37:19.285382+0000","last_clean":"2024-09-21T23:37:19.285382+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:19.285382+0000","last_undegraded":"2024-09-21T23:37:19.285382+0000","last_fullsized":"2024-09-21T23:37:19.285382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.296725+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:19.296725+0000","last_peered":"2024-09-21T23:37:19.296725+0000","last_clean":"2024-09-21T23:37:19.296725+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:19.296725+0000","last_undegraded":"2024-09-21T23:37:19.296725+0000","last_fullsized":"2024-09-21T23:37:19.296725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.278536+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:19.278536+0000","last_peered":"2024-09-21T23:37:19.278536+0000","last_clean":"2024-09-21T23:37:19.278536+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:19.278536+0000","last_undegraded":"2024-09-21T23:37:19.278536+0000","last_fullsized":"2024-09-21T23:37:19.278536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.282766+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:19.282766+0000","last_peered":"2024-09-21T23:37:19.282766+0000","last_clean":"2024-09-21T23:37:19.282766+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:19.282766+0000","last_undegraded":"2024-09-21T23:37:19.282766+0000","last_fullsized":"2024-09-21T23:37:19.282766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.276165+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:19.276165+0000","last_peered":"2024-09-21T23:37:19.276165+0000","last_clean":"2024-09-21T23:37:19.276165+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:19.276165+0000","last_undegraded":"2024-09-21T23:37:19.276165+0000","last_fullsized":"2024-09-21T23:37:19.276165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336300,"statfs":{"total":96636764160,"available":96600371200,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:23.162 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:37:23.162 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T23:37:23.162 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T23:37:23.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T23:37:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:23.460 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T23:37:23.477 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T23:37:23.477 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T23:37:23.486 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T23:37:23.486 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:37:23.486 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T23:37:23.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T23:37:23.789 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T23:37:23.790 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:37:23.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:24.082 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:24.098 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T23:37:22.952543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":189440,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566041600,"statfs":{"total":579820584960,"available":579626598400,"internally_reserved":0,"allocated":2326528,"data_stored":1561900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.694850"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277603+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:19.277603+0000","last_peered":"2024-09-21T23:37:19.277603+0000","last_clean":"2024-09-21T23:37:19.277603+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:19.277603+0000","last_undegraded":"2024-09-21T23:37:19.277603+0000","last_fullsized":"2024-09-21T23:37:19.277603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272836+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:19.272836+0000","last_peered":"2024-09-21T23:37:19.272836+0000","last_clean":"2024-09-21T23:37:19.272836+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:19.272836+0000","last_undegraded":"2024-09-21T23:37:19.272836+0000","last_fullsized":"2024-09-21T23:37:19.272836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277632+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:19.277632+0000","last_peered":"2024-09-21T23:37:19.277632+0000","last_clean":"2024-09-21T23:37:19.277632+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:19.277632+0000","last_undegraded":"2024-09-21T23:37:19.277632+0000","last_fullsized":"2024-09-21T23:37:19.277632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272996+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:19.272996+0000","last_peered":"2024-09-21T23:37:19.272996+0000","last_clean":"2024-09-21T23:37:19.272996+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:19.272996+0000","last_undegraded":"2024-09-21T23:37:19.272996+0000","last_fullsized":"2024-09-21T23:37:19.272996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.285382+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:19.285382+0000","last_peered":"2024-09-21T23:37:19.285382+0000","last_clean":"2024-09-21T23:37:19.285382+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:19.285382+0000","last_undegraded":"2024-09-21T23:37:19.285382+0000","last_fullsized":"2024-09-21T23:37:19.285382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.296725+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:19.296725+0000","last_peered":"2024-09-21T23:37:19.296725+0000","last_clean":"2024-09-21T23:37:19.296725+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:19.296725+0000","last_undegraded":"2024-09-21T23:37:19.296725+0000","last_fullsized":"2024-09-21T23:37:19.296725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.278536+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:19.278536+0000","last_peered":"2024-09-21T23:37:19.278536+0000","last_clean":"2024-09-21T23:37:19.278536+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:19.278536+0000","last_undegraded":"2024-09-21T23:37:19.278536+0000","last_fullsized":"2024-09-21T23:37:19.278536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.282766+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:19.282766+0000","last_peered":"2024-09-21T23:37:19.282766+0000","last_clean":"2024-09-21T23:37:19.282766+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:19.282766+0000","last_undegraded":"2024-09-21T23:37:19.282766+0000","last_fullsized":"2024-09-21T23:37:19.282766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.276165+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:19.276165+0000","last_peered":"2024-09-21T23:37:19.276165+0000","last_clean":"2024-09-21T23:37:19.276165+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:19.276165+0000","last_undegraded":"2024-09-21T23:37:19.276165+0000","last_fullsized":"2024-09-21T23:37:19.276165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336300,"statfs":{"total":96636764160,"available":96600371200,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:24.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:24.377 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:24.378 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:24.394 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T23:37:22.952543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":189440,"kb_used_data":2272,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566041600,"statfs":{"total":579820584960,"available":579626598400,"internally_reserved":0,"allocated":2326528,"data_stored":1561900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.694850"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277603+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:19.277603+0000","last_peered":"2024-09-21T23:37:19.277603+0000","last_clean":"2024-09-21T23:37:19.277603+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:19.277603+0000","last_undegraded":"2024-09-21T23:37:19.277603+0000","last_fullsized":"2024-09-21T23:37:19.277603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272836+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:19.272836+0000","last_peered":"2024-09-21T23:37:19.272836+0000","last_clean":"2024-09-21T23:37:19.272836+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:19.272836+0000","last_undegraded":"2024-09-21T23:37:19.272836+0000","last_fullsized":"2024-09-21T23:37:19.272836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.277632+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:19.277632+0000","last_peered":"2024-09-21T23:37:19.277632+0000","last_clean":"2024-09-21T23:37:19.277632+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:19.277632+0000","last_undegraded":"2024-09-21T23:37:19.277632+0000","last_fullsized":"2024-09-21T23:37:19.277632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.272996+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:19.272996+0000","last_peered":"2024-09-21T23:37:19.272996+0000","last_clean":"2024-09-21T23:37:19.272996+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:19.272996+0000","last_undegraded":"2024-09-21T23:37:19.272996+0000","last_fullsized":"2024-09-21T23:37:19.272996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.285382+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:19.285382+0000","last_peered":"2024-09-21T23:37:19.285382+0000","last_clean":"2024-09-21T23:37:19.285382+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:19.285382+0000","last_undegraded":"2024-09-21T23:37:19.285382+0000","last_fullsized":"2024-09-21T23:37:19.285382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.296725+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:19.296725+0000","last_peered":"2024-09-21T23:37:19.296725+0000","last_clean":"2024-09-21T23:37:19.296725+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:19.296725+0000","last_undegraded":"2024-09-21T23:37:19.296725+0000","last_fullsized":"2024-09-21T23:37:19.296725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.278536+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:19.278536+0000","last_peered":"2024-09-21T23:37:19.278536+0000","last_clean":"2024-09-21T23:37:19.278536+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:19.278536+0000","last_undegraded":"2024-09-21T23:37:19.278536+0000","last_fullsized":"2024-09-21T23:37:19.278536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.282766+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:19.282766+0000","last_peered":"2024-09-21T23:37:19.282766+0000","last_clean":"2024-09-21T23:37:19.282766+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:19.282766+0000","last_undegraded":"2024-09-21T23:37:19.282766+0000","last_fullsized":"2024-09-21T23:37:19.282766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T23:37:19.276165+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:19.276165+0000","last_peered":"2024-09-21T23:37:19.276165+0000","last_clean":"2024-09-21T23:37:19.276165+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:19.276165+0000","last_undegraded":"2024-09-21T23:37:19.276165+0000","last_fullsized":"2024-09-21T23:37:19.276165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336300,"statfs":{"total":96636764160,"available":96600371200,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":63521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":192512,"data_stored":63540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":653889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:24.395 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:37:24.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T23:37:24.677 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 21 2024-09-21T23:37:24.677 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 66973394-6d88-4f14-bf11-7ccd7c9cf4a7 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-09-21T23:36:58.086363+0000 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2024-09-21T23:37:22.820320+0000 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:crush_version 5 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:full_ratio 0.95 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:backfillfull_ratio 0.9 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:nearfull_ratio 0.85 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:require_min_compat_client luminous 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:min_compat_client jewel 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:require_osd_release squid 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:stretch_mode_enabled false 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-21T23:37:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 20 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:max_osd 6 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/915397992,v1:172.21.15.79:6801/915397992] [v2:172.21.15.79:6802/915397992,v1:172.21.15.79:6803/915397992] exists,up 638e3e48-1763-4ec9-8c06-634e0b3b793b 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3900686792,v1:172.21.15.79:6809/3900686792] [v2:172.21.15.79:6810/3900686792,v1:172.21.15.79:6811/3900686792] exists,up 02967526-d5c5-4f68-8e25-343311dbaec4 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2564679849,v1:172.21.15.79:6817/2564679849] [v2:172.21.15.79:6818/2564679849,v1:172.21.15.79:6819/2564679849] exists,up 4f770ac1-d062-4934-81d5-7f980b3830a1 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/1410364236,v1:172.21.15.118:6809/1410364236] [v2:172.21.15.118:6810/1410364236,v1:172.21.15.118:6811/1410364236] exists,up 90cc3313-3845-48d0-839d-8fea4c52bfde 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/1572346429,v1:172.21.15.118:6801/1572346429] [v2:172.21.15.118:6802/1572346429,v1:172.21.15.118:6803/1572346429] exists,up 42b8e641-af88-4d41-bc5f-a14020f205ce 2024-09-21T23:37:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] [v2:172.21.15.118:6818/466572374,v1:172.21.15.118:6819/466572374] exists,up 2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9 2024-09-21T23:37:24.694 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/915397992,v1:172.21.15.79:6801/915397992] [v2:172.21.15.79:6802/915397992,v1:172.21.15.79:6803/915397992] exists,up 638e3e48-1763-4ec9-8c06-634e0b3b793b', 'osd.1 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3900686792,v1:172.21.15.79:6809/3900686792] [v2:172.21.15.79:6810/3900686792,v1:172.21.15.79:6811/3900686792] exists,up 02967526-d5c5-4f68-8e25-343311dbaec4', 'osd.2 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2564679849,v1:172.21.15.79:6817/2564679849] [v2:172.21.15.79:6818/2564679849,v1:172.21.15.79:6819/2564679849] exists,up 4f770ac1-d062-4934-81d5-7f980b3830a1', 'osd.3 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/1410364236,v1:172.21.15.118:6809/1410364236] [v2:172.21.15.118:6810/1410364236,v1:172.21.15.118:6811/1410364236] exists,up 90cc3313-3845-48d0-839d-8fea4c52bfde', 'osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/1572346429,v1:172.21.15.118:6801/1572346429] [v2:172.21.15.118:6802/1572346429,v1:172.21.15.118:6803/1572346429] exists,up 42b8e641-af88-4d41-bc5f-a14020f205ce', 'osd.5 up in weight 1 up_from 12 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] [v2:172.21.15.118:6818/466572374,v1:172.21.15.118:6819/466572374] exists,up 2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9'] 2024-09-21T23:37:24.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mon.a", 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster": "ceph", 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket_mode": "", 2024-09-21T23:37:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_client_required": "cephx, none", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_cluster_required": "cephx", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_debug": "true", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_required": "cephx", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_supported": "cephx", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio": "true", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_reap_max": "16", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_async_discard": "false", 2024-09-21T23:37:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_block_size": "4096", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_enable_discard": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry": "3", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash": "0", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_hipri": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T23:37:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_type": "", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_allocator": "hybrid", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_buffered_io": "true", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T23:37:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery": "false", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_sync_write": "false", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T23:37:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocator": "hybrid", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_create": "true", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_create": "false", 2024-09-21T23:37:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_path": "", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_size": "0", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_path": "", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_size": "107374182400", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_create": "false", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_path": "", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs": "true", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:37:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune": "true", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size": "0", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_type": "2q", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_clone_cow": "true", 2024-09-21T23:37:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_mode": "none", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_freelist": "false", 2024-09-21T23:37:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T23:37:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fail_eio": "false", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T23:37:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T23:37:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size": "0", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T23:37:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T23:37:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_mem": "512", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_tracing": "false", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T23:37:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T23:37:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_version": "2", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_signatures": "false", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_version": "2", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_signatures": "false", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_version": "2", 2024-09-21T23:37:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_sign_messages": "true", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "chdir": "", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_acl_type": "", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_asio_thread_count": "2", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_mid": "0.750000", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_size": "16384", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_caps_release_delay": "5", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_check_pool_perm": "true", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_force_sync_read": "false", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_getattr_caps": "false", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_features": "", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T23:37:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_remount": "false", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_dirsize_rbytes": "true", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_force_lazyio": "false", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_fs": "", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_release_failure": "false", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_inline_size": "4096", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mds_namespace": "", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_metadata": "", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_gid": "-1", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_timeout": "300", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_uid": "-1", 2024-09-21T23:37:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mountpoint": "/", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_notify_timeout": "10", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc": "true", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_objects": "1000", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_size": "209715200", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_permissions": "true", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota": "true", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota_df": "true", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_bytes": "0", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_periods": "4", 2024-09-21T23:37:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_min": "131072", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_reconnect_stale": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_shutdown_timeout": "30", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_snapdir": ".snap", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_tick_interval": "1", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_trace": "", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_faked_inos": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_random_mds": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_port": "12201", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_monitors": "default=true", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog": "false", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T23:37:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_level": "info", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "-", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network": "", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network_interface": "", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_isal": "false", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_level": "5", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zstd_level": "1", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T23:37:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook": "", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook_timeout": "10", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "daemonize": "false", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "daos_pool": "tank", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_alienstore": "0/5", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T23:37:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok": "1/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok_assert_abort": "false", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_auth": "20/20", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bdev": "1/3", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluefs": "1/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluestore": "1/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_buffer": "0/1", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephsqlite": "0/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_client": "0/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_compressor": "1/5", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_context": "0/1", 2024-09-21T23:37:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crush": "1/1", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crypto": "1/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cyanstore": "0/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_dpdk": "1/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_eventtrace": "1/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filer": "0/1", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filestore": "1/3", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_finisher": "1/1", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_fuse": "1/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T23:37:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_javaclient": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journal": "1/3", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journaler": "0/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_kstore": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_lockdep": "0/1", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mclock": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_balancer": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_locker": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_migrator": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_memstore": "1/5", 2024-09-21T23:37:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgr": "2/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgrc": "1/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mon": "20/20", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_monc": "0/10", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ms": "1/1", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_none": "0/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objclass": "0/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objectcacher": "0/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objecter": "0/1", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_optracker": "0/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_osd": "1/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_paxos": "20/20", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_perfcounter": "1/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_prioritycache": "1/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rados": "0/5", 2024-09-21T23:37:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd": "0/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_replay": "0/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_refs": "0/0", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_reserver": "1/1", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_access": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_flight": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_sync": "1/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rocksdb": "4/5", 2024-09-21T23:37:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_backref": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cache": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_device": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_epm": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_journal": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_lba": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_odata": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_omap": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_onode": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_t": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_tm": "0/5", 2024-09-21T23:37:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_striper": "0/1", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_test": "0/5", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_throttle": "1/1", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_timer": "0/1", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_tp": "0/5", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_trace": "1/5", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "device_failure_prediction_mode": "none", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_graylog": "false", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_journald": "false", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_stderr": "true", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_syslog": "false", 2024-09-21T23:37:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event_tracing": "false", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_http_port": "9926", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_prio_limit": "5", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sort_metrics": "true", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_stats_period": "5", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "fake_statfs_for_testing": "0", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "fatal_signal_handlers": "true", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_purge_ops": "10", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_truncate_ops": "128", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_blackhole": "false", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_snap": "true", 2024-09-21T23:37:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_caller_concurrency": "10", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_omap_check": "false", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_verify_split": "false", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_dump_file": "", 2024-09-21T23:37:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fadvise": "true", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fail_eio": "true", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_size": "128", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap": "false", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_inject_stall": "0", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_parallel": "false", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_trailing": "false", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_writeahead": "false", 2024-09-21T23:37:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_kill_at": "0", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T23:37:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_merge_threshold": "-10", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_odsync_write": "false", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend_path": "", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_threads": "2", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_punch_hole": "false", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T23:37:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_ops": "50", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_seek_data_hole": "false", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc": "false", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_splice": "false", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_multiple": "2", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_rand_factor": "20", 2024-09-21T23:37:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_update_to": "1000", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_xfs_extsize": "false", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_zfs_snap": "false", 2024-09-21T23:37:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "fio_dir": "/tmp/fio", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66973394-6d88-4f14-bf11-7ccd7c9cf4a7", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_allow_other": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_big_writes": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_debug": "false", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_default_permissions": "false", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_disable_pagecache": "false", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_max_write": "0", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_multithreaded": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_require_active_mds": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_set_user_groups": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_move": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_read": "true", 2024-09-21T23:37:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_write": "true", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_target_name": "ceph", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_file": "", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_inject_failure": "0", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_interval": "5", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "host": "smithi079", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T23:37:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "inject_early_sigterm": "false", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_agent_port": "6799", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_tracing_enable": "false", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_aio": "true", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_align_min_size": "65536", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_align": "true", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_size": "4096", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_dio": "true", 2024-09-21T23:37:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_discard": "false", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_force_aio": "false", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_ignore_corruption": "false", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_entries": "100", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_replay_from": "0", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_write_header_frequency": "0", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_zero_on_create": "false", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prefetch_periods": "10", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prezero_periods": "5", 2024-09-21T23:37:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_write_head_interval": "15", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "keyfile": "", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_backend": "rocksdb", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_bytes": "67108864", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_ops": "512", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_onode_map_size": "1024", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T23:37:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_transaction": "false", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "librados_thread_count": "2", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep": "false", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep_force_backtrace": "false", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_coarse_timestamps": "true", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_flush_on_exit": "false", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_port": "12201", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_new": "1000", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_recent": "10000", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stderr_prefix": "", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T23:37:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": "true", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_graylog": "false", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_journald": "false", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_stderr": "false", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_syslog": "false", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_action_on_write_error": "1", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_alternate_name_max": "8192", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_asio_thread_count": "2", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_export_pin": "true", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T23:37:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_interval": "10", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max": "-1", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max_until": "-1", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_merge_size": "50", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T23:37:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_mode": "0", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_bits": "3", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_size": "10000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T23:37:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_mid": "0.700000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_interval": "1", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T23:37:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_auth_pins": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_frag": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_scatterstat": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_subtrees": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_default_dir_hash": "2", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_defer_session_stale": "true", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T23:37:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_entries": "0", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_prefetch": "true", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_early_reply": "true", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enable_op_tracker": "true", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enforce_unique_name": "true", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T23:37:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_health_dummy": "false", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T23:37:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_join_fs": "", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_journal_format": "1", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_create_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_export_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_import_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_link_at": "0", 2024-09-21T23:37:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_openc_at": "0", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_rename_at": "0", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_events": "-1", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_segments": "128", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_pause": "false", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_segment_size": "0", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T23:37:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_trim_threshold": "128", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_requests": "100000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_export_size": "20971520", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_file_recover": "32", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_files": "64", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops": "8192", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T23:37:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_metrics_update_interval": "2", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_per_client": "100", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_numa_node": "-1", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_duration": "600", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_size": "20", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_log_threshold": "5", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_grace": "15", 2024-09-21T23:37:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_interval": "5", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_caps": "30000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_interval": "1.000000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_gid": "0", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_uid": "0", 2024-09-21T23:37:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_shutdown_check": "0", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_skip_ino": "0", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T23:37:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_min_uid": "0", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_rstat": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_symlink_recovery": "true", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_exports": "0", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_fragments": "0", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_tick_interval": "5.000000", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_valgrind_exit": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_backtrace": "1", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_scatter": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_sessions": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool_debug": "false", 2024-09-21T23:37:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_set": "false", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_size": "65536", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_bytes": "134217728", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_messages": "512", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_disabled_modules": "", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T23:37:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_messages": "128", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_messages": "128", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_history_duration": "600", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_history_size": "20", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_op_log_threshold": "5", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_messages": "8192", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_pool": "true", 2024-09-21T23:37:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_standby_modules": "true", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_period": "5", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_threshold": "5", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_tick_period": "2", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_delete": "true", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_bytes": "104857600", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T23:37:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_target_rank": "-1", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T23:37:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_start": "false", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_trim": "true", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T23:37:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cpu_threads": "4", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_crit": "5", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_warn": "5", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_json": "false", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_extra_checks": "false", 2024-09-21T23:37:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_down_uptime_grace": "60", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_default_strategy": "1", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_timeout": "5.000000", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T23:37:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_enable_op_tracker": "true", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_fake_pool_delete": "false", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_force_quorum_join": "false", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_log_update_period": "5", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_max_detail": "50", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog": "true", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host": "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.118", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host_override": "", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_initial_members": "", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T23:37:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease": "5.000000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_full_interval": "50", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max": "10000", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max_summary": "50", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_epochs": "500", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T23:37:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_osd": "10000", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_autotune": "true", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_target": "2147483648", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_digest_period": "5", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T23:37:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_complaint_time": "30", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_duration": "600", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_size": "20", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_log_threshold": "5", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T23:37:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size": "500", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T23:37:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_report_timeout": "900", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T23:37:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T23:37:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_osds": "4", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_interval": "86400", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_max_keys": "100", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_timeout": "300", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_session_timeout": "300", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_smart_report_timeout": "5", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_size": "4", 2024-09-21T23:37:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_debug": "false", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_tick_interval": "5", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T23:37:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_older_version": "true", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T23:37:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": "", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_tracing_enabled": "false", 2024-09-21T23:37:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_op_threads": "3", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_cm": "false", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_device_name": "", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_sl": "3", 2024-09-21T23:37:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_type": "ib", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_reap_threshold": "5", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_before_connect": "false", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv4": "true", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv6": "false", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr1": "true", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr2": "true", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_max": "7568", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_min": "6800", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_count": "3", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_delay": "5", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_client": "false", 2024-09-21T23:37:24.885 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mds": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mgr": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mon": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_osd": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_client_mode": "crc secure", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_type": "", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_compress_secure": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_data": "true", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_header": "true", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bug": "true", 2024-09-21T23:37:24.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_old_message": "true", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hugepages": "", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_lro": "true", 2024-09-21T23:37:24.887 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_pmd": "", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_port_id": "0", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_on_send": "false", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_type": "", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_network_congestion": "0", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T23:37:24.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_accept_failures": "4", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_backoff": "15.000000", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_mode": "none", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_min_cost": "65536", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_public_type": "", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_service_mode": "crc secure", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_nodelay": "true", 2024-09-21T23:37:24.889 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_type": "async+posix", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "no_config_file": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_ops": "1024", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_timeout": "10.000000", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objectstore_blackhole": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T23:37:24.890 INFO:teuthology.orchestra.run.smithi079.stdout: "openssl_engine_opts": "", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_ops": "4", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_slop": "0.020000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_asio_thread_count": "2", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T23:37:24.891 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_max": "512", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_min": "64", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_peering": "false", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_beacon_report_interval": "300", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_duration": "30", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blkin_trace_all": "false", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T23:37:24.892 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_update_on_start": "true", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_cap": "256", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_op_priority": "63", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_watch_timeout": "30", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_command_max_records": "256", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_compact_on_start": "false", 2024-09-21T23:37:24.893 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_on_start": "true", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T23:37:24.894 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_op_order": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_shutdown": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_trim_objects": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T23:37:24.895 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_notify_timeout": "30", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T23:37:24.896 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_enable_op_tracker": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_info": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_function_tracing": "false", 2024-09-21T23:37:24.897 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_grace": "20", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_interval": "6", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_stale": "600", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T23:37:24.898 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_size": "5120", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_kill_backfill_at": "0", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_cache_size": "50", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_dedup": "true", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max": "40", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_name_len": "100", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_size": "0", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_backfills": "1", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_count": "5", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_period": "600", 2024-09-21T23:37:24.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_name_len": "2048", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_size": "134217728", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pgls": "1024", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_cost": "8388608", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_objects": "10", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_scrubs": "3", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T23:37:24.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_size": "90", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_profile": "balanced", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T23:37:24.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_base": "805306368", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target": "4294967296", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_autotune": "false", 2024-09-21T23:37:24.902 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_recovery_priority": "0", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_interval": "5", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_cache_shards": "32", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_node": "-1", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T23:37:24.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objecter_finishers": "1", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore": "bluestore", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_fuse": "false", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_tracing": "false", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_duration": "600", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_size": "20", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_log_threshold": "5", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards": "0", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T23:37:24.904 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_timeout": "15", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_open_classes_on_start": "true", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_os_flags": "0", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_peering_op_priority": "255", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_priority": "5", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T23:37:24.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crimson": "false", 2024-09-21T23:37:24.906 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flags": "0", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_min_size": "0", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T23:37:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_size": "2", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_type": "replicated", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_cost": "20971520", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T23:37:24.908 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active": "0", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_priority": "3", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_priority": "5", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_repair_during_recovery": "false", 2024-09-21T23:37:24.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_cost": "52428800", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T23:37:24.910 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_hour": "0", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_priority": "5", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T23:37:24.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_skip_data_digest": "false", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_smart_report_timeout": "5", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_priority": "5", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T23:37:24.912 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_transaction_size": "30", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tracing": "false", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_use_stale_snap": "false", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T23:37:24.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_kill_at": "0", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_max_join_drift": "10", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min": "500", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min_wait": "0.050000", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max": "500", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_min": "250", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_stash_full_interval": "25", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_max": "500", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_min": "250", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "perf": "true", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T23:37:24.914 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "-", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrv": "", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "public_bind_addr": "-", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network": "", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network_interface": "", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_enabled": "false", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_mon_op_timeout": "0", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_osd_op_timeout": "0", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_tracing": "false", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_atime_update_interval": "60", 2024-09-21T23:37:24.915 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache": "true", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_policy": "writearound", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_size": "33554432", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T23:37:24.916 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_compression_hint": "none", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_clone_format": "auto", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_data_pool": "", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_features": "61", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_format": "2", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_map_options": "", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_order": "22", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_pool": "rbd", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_count": "0", 2024-09-21T23:37:24.917 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler": "simple", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T23:37:24.918 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_order": "24", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_pool": "", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_splay_width": "4", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T23:37:24.919 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T23:37:24.920 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T23:37:24.921 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_threads": "1", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_plugins": "", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T23:37:24.922 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T23:37:24.923 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_tracing": "false", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_names": "true", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_pool": "true", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_base_url": "", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_log_level": "", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T23:37:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_admin_entry": "admin", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_backend_store": "rados", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_barbican_url": "", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_beast_enable_async": "true", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T23:37:24.925 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_enabled": "true", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_config_store": "rados", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_content_length_compat": "false", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cross_domain_policy": "", 2024-09-21T23:37:24.926 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T23:37:24.927 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T23:37:24.928 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T23:37:24.929 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d4n_port": "6379", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_window": "30", 2024-09-21T23:37:24.930 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T23:37:24.931 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_name": "", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_s3website_name": "", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T23:37:24.932 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_ops_log": "false", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_static_website": "false", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_usage_log": "false", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_expose_bucket": "false", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_extended_http_attrs": "", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_filter": "none", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T23:37:24.933 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_objs": "32", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T23:37:24.934 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_init_timeout": "300", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T23:37:24.935 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_project": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_user": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_api_version": "2", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T23:37:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_url": "", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_objs": "32", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_worker": "3", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T23:37:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_uri": "", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_http_headers": "", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T23:37:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_size": "0", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_listing_results": "1000", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_notify_retries": "10", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T23:37:24.939 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T23:37:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_control_oids": "8", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_numa_node": "-1", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T23:37:24.941 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_tracing": "false", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_token": "", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_url": "", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_rados": "false", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T23:37:24.942 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_continue": "true", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T23:37:24.943 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_tracing": "false", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm": "", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_id": "", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region": "", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_request_uri": "", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T23:37:24.944 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_resolve_cname": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_run_sync_thread": "true", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T23:37:24.945 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_script_uri": "", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_service_provider_name": "", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_id": "", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_secret": "", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_entry": "sts", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_key": "sts", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T23:37:24.946 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_url": "", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_custom_header": "", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_need_stats": "true", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_tenant_name": "", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url": "", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_lease_period": "120", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T23:37:24.947 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_thread_pool_size": "512", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_comment": "", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_createby": "", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_encoding": "", 2024-09-21T23:37:24.948 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_flag": "false", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_origin": "", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_tracker": "", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_shards": "32", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_use_opa_authz": "false", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_counters_cache": "false", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T23:37:24.949 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_unique_email": "true", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_verify_ssl": "true", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone": "", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_id": "", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup": "", 2024-09-21T23:37:24.950 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_id": "", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_block_size": "4096", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T23:37:24.951 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_partition_filters": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_perf": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "run_dir": "/var/run/ceph", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_block_create": "true", 2024-09-21T23:37:24.952 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_device_size": "53687091200", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_full_integrity_check": "false", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_main_device_type": "SSD", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T23:37:24.953 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_segment_size": "67108864", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "set_keepcaps": "false", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "setgroup": "", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser": "", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser_match_path": "", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "thp": "false", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_default_timeout": "60", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T23:37:24.954 INFO:teuthology.orchestra.run.smithi079.stdout: "throttler_perf_counter": "true" 2024-09-21T23:37:24.955 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:24.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T23:37:25.095 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-21T23:37:25.095 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:37:25.107 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-21T23:37:25.107 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:37:25.116 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-21T23:37:25.116 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:37:25.131 DEBUG:teuthology.orchestra.run.smithi118:> type ceph-objectstore-tool 2024-09-21T23:37:25.136 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:37:25.136 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2024-09-21T23:37:25.140 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:37:25.140 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T23:37:25.162 INFO:tasks.workunit:Pulling workunits from ref 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:37:25.162 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T23:37:25.162 DEBUG:teuthology.orchestra.run.smithi165:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:37:25.163 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:37:25.164 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T23:37:25.164 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T23:37:25.164 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T23:37:25.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:25.165 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T23:37:25.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:25.165 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T23:37:25.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:37:25.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:37:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T23:37:25.171 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-21T23:37:25.186 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:25.186+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:25.220 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-21T23:37:25.220 DEBUG:teuthology.orchestra.run.smithi165:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-21T23:37:25.266 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:25.263+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:25.267 INFO:tasks.workunit:timeout=3h 2024-09-21T23:37:25.267 INFO:tasks.workunit:cleanup=True 2024-09-21T23:37:25.268 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:37:25.299 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:25.299 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:25.305 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:37:25.305 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:25.314 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:25.315 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:25.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:25.318 INFO:tasks.workunit.client.0.smithi165.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T23:37:25.335 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:25.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:25.351 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:25.352 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:25.367 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:25.366+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:25.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:25.373 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:25.393 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:25.393 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:25.414 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:25.414 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:25.437 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:25.437 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:25.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:25.467 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:25.467+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:25.568 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:25.566+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:25.600 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:25.600 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:25.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:25.698 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:25.698+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:25.770 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:25.771+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:25.774 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:25.774 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:25.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:25.874 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:25.874+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:25.945 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:25.945 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:25.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:26.002 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:26.002+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:26.073 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:26.070+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:26.123 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.123 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:26.178 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:26.178+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:26.275 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:26.279+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:26.296 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.296 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:26.368 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:37:26.375 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:26.374+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:26.430 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.430 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:26.486 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:26.486+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:26.579 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:26.578+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:26.608 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.608 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.622 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:26.679 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:26.678+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:26.753 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.753 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.766 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:26.780 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:26.778+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:26.881 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:26.879+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:26.885 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:26.885 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:26.898 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:26.986 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:26.987+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:27.066 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:27.067 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:27.082 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:27.130 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:27.131+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:27.184 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:27.182+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:27.254 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:27.255 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:27.268 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:27.285 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:27.282+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:27.386 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:27.387+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:27.391 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:27.392 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:27.406 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:27.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:27.494 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:27.495+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:27.567 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:27.567 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:27.581 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:27.589 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:27.589 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:27.607 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:27.607 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:27.622 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:27.623+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:27.624 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:27.624 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:27.643 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:27.644 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:27.661 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:27.662 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:27.680 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:27.680 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:27.688 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:27.686+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:27.744 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:27.744 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:27.744 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:27.744 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:27.758 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:27.802 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:27.803+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:27.875 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:27.875 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:27.889 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:27.930 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:27.931+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:27.991 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:27.990+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:28.054 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:28.054 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:28.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:28.054 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:28.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.790951830000001, 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:28.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:28.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.784194809000001, 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:28.061 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.778871150000001, 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.065 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:28.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.77800354, 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:28.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7938054829999999, 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:28.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:28.078 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7911042540000004, 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:28.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:28.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7793683429999998, 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:28.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7726157130000004, 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:28.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:28.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:28.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:28.093 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:28.094 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:28.095+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:28.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:28.195 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:28.195+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:28.225 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:28.225 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:28.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:28.296 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:28.299+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:28.361 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:28.362 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:28.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:28.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:37:28.397 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:28.395+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:28.498 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:28.495+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:28.503 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.532334731000001, 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.497179293, 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.483708563, 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.460034466, 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.396707505, 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.224537510999999, 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.873898216000001, 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.873190428000001, 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.872642919, 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:28.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.871990198000001, 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.871034588000001, 2024-09-21T23:37:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.87037789, 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.869846133999999, 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.869032474000001, 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.868560571, 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.867530939, 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.86690829, 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.861322551000001, 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.860362281, 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:28.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:28.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.859601446999999, 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:28.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:28.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:28.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:28.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:28.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:28.638 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:28.638+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:28.699 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:28.698+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:28.713 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:28.714 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:28.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:28.800 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:28.803+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:28.894 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:28.895 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:28.901 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:28.903+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:28.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:29.002 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:29.002+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:29.029 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.769545831, 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:37:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7514579050000005, 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:37:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:37:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:37:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:29.039 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:29.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:29.104 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:29.103+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:29.206 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:29.202+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:29.206 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:29.206 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:29.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:29.306 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:29.306+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:29.379 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:37:29.389 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:29.389 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:29.406 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:29.403+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:29.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:29.506 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:29.507+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:29.568 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.301516521, 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:29.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:29.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:29.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:29.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:29.573 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:29.582 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:29.626 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:29.627+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:29.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:29.711 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:29.711 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:29.711 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:29.712 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:29.725 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:29.746 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:29.746+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:29.808 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:29.806+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:29.844 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:29.845 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:29.861 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:29.861 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:29.877 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:29.877 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:29.891 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:29.891 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:29.891 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:29.892 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:29.892 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:29.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:29.897 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:29.906 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:29.908 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:29.906+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:29.918 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:29.918 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:29.937 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:29.937 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:30.010 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:30.007+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:30.024 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.024 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.038 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:30.134 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:30.135+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:30.204 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.204 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.211 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:30.211+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:30.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:30.312 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:30.310+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:30.319 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:30.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:30.319 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T23:37:30.319 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:37:30.319 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:37:30.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T23:37:30.386 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.386 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.401 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:30.412 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:30.410+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:30.440 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:37:30.440 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:30.513 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:30.511+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:30.519 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.520 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.534 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:30.614 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:30.614+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:30.694 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.695 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:30.714 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:30.714+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:30.816 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:30.814+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:30.831 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.831 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.845 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:30.916 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:30.914+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:30.967 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:30.967 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:30.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:31.017 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:31.015+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:31.113 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.113 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.118 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:31.119+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:31.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:31.219 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:31.218+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:31.245 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.245 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:31.320 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:31.319+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:31.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:37:31.430 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.430 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.431 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:31.430+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:31.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:31.534 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:31.534+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:31.621 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.621 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.623 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:31.627+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:31.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:31.723 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:31.723+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:31.854 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:31.854+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:31.925 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:31.922+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:31.929 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:31.929 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:31.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:31.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:32.034 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:32.034+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:32.110 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:32.110 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:32.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:32.127 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:32.131+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:32.137 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:32.137 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:32.154 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:32.154 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:32.185 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:32.185 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:32.205 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:32.206 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:32.226 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:32.227 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:32.226+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:32.246 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:32.246 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:32.261 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:32.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:32.261 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:32.261 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:32.262 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:32.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:32.362 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:32.362+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:32.396 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.167210719, 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:32.439 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:32.439 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:32.434+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:32.447 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:32.530 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:32.530+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:32.579 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:32.579 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:32.579 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:32.579 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:32.593 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:32.631 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:32.630+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:32.709 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:32.709 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:32.709 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:32.709 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:32.709 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:32.722 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:32.733 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:32.730+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:32.834 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:32.830+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 20.613649573, 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:32.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:32.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:32.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.844464118000001, 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:32.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:32.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.856 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.844332595000001, 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:32.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:32.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.844272686, 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.863 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.74344541, 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:32.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:32.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.743329921000001, 2024-09-21T23:37:32.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:32.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:32.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.577058208, 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.880 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:32.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.577011297999999, 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:32.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:32.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576969911999999, 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:32.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576793039999998, 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:32.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:32.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:32.903 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576745343999999, 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.904 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:32.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576385160000001, 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:32.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:32.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:32.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576257204000001, 2024-09-21T23:37:32.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:32.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:32.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.576219928, 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:32.921 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:32.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:32.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.212460523000001, 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.927 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:32.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.212425762999999, 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:32.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.938 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.212329836999999, 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:32.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:32.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.597608387000001, 2024-09-21T23:37:32.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:32.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.597549530999999, 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.950 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:32.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:32.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.597436338000001, 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:32.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:32.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:32.961 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:32.963 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:32.963 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:32.967+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:33.063 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:33.063+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:33.136 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:33.136 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:33.136 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:33.136 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:33.150 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:33.194 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:33.199+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:33.265 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:33.262+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:33.318 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:33.318 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:33.318 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:33.318 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:33.319 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:33.332 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:33.382 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:33.383+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:33.451 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:33.451 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:33.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:33.451 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:37:33.451 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:33.467 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:33.467 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:33.466+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:33.567 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:33.567+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:33.635 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:33.635 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:33.635 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:33.635 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:33.648 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:33.694 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:33.695+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:33.769 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:33.767+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:33.818 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:33.819 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:33.832 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:33.870 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:33.870+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:33.971 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:33.970+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 16.736692342000001, 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.000 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:34.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:34.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:34.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 16.729935320999999, 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:34.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:34.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 16.724611662000001, 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:34.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:34.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:34.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 16.723744052000001, 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:34.015 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:34.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.739545995, 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:34.020 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:34.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:34.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:34.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.025 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.736844766000001, 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:34.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.725108855, 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:34.030 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:34.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.718356225000001, 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:34.037 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:34.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:34.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:34.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:34.041 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:34.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:34.090 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:34.090+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:34.164 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:34.164 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:34.164 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:34.164 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:34.173 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:34.175+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:34.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:34.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:34.274 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:34.271+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:34.339 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:34.339 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:34.339 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:34.339 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:34.339 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:34.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:34.398 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:34.398+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:34.410 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:34.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:34.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:37:34.426 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:34.427 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:34.443 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:34.443 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:34.464 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:34.464 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:34.465 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:34.465 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:34.476 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:34.474+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:34.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:34.485 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:34.485 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:34.505 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:34.505 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:34.577 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:34.574+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:34.592 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:34.593 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:34.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:34.678 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:34.679+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:34.756 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:34.756 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:34.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:34.778 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:34.783+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:34.879 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:34.878+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:34.891 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:34.891 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:34.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:34.990 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:34.990+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:35.065 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.065 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:35.081 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:35.079+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:35.181 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:35.183+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:35.245 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.245 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:35.306 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:35.306+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:35.378 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.379 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.382 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:35.382+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:35.390 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:35.413 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:37:35.455 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:35.456 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:35.456 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:37:35.456 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-21T23:37:35.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T23:37:35.483 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:35.483+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:35.519 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.519 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.534 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:35.626 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:35.627+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:35.685 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:35.683+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:35.701 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.701 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.715 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:35.786 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:35.786+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:35.879 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:35.879 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:35.887 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:35.887+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:35.893 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:35.990 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:35.986+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:36.063 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:36.063 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:36.077 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:36.089 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:36.086+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:36.190 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:36.186+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:36.198 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:36.198 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:36.213 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:36.291 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:36.290+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:36.336 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:36.337 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:36.350 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:36.394 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:36.390+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:36.412 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T23:37:36.475 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:36.475 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:36.489 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:36.493 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:36.490+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:36.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:36.594 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:36.595+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:36.610 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:36.610 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:36.624 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:36.677 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:36.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:36.694 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:36.694 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:36.694 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:36.694+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:36.710 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:36.710 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:36.729 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:36.730 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:36.747 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:36.748 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:36.768 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:36.768 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:36.795 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:36.795 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:36.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.532459800000002, 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:36.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.525702779, 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:36.802 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:36.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:36.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.806 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.520379120000001, 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:36.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:36.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.519511510000001, 2024-09-21T23:37:36.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:36.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:36.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.535313453000001, 2024-09-21T23:37:36.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:36.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:36.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:36.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.532612224000001, 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.822 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:36.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.520876312999999, 2024-09-21T23:37:36.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:36.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:36.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:36.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.514123683000001, 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:36.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:36.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:36.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:36.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:36.837 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:36.837 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:36.830+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:36.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:36.895 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:36.894+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:36.963 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:36.964 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:36.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:37.018 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:37.018+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:37.097 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:37.094+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:37.142 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:37.142 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:37.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:37.198 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:37.203+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:37.299 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:37.298+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:37.327 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.354993916000002, 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:37.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:37.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.319838478000001, 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.306367748, 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:37.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:37.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:37.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.282693650999999, 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.219366690000001, 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.047196696, 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:37.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.696557401, 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:37.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:37.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.695849613, 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:37.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:37.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.695302104, 2024-09-21T23:37:37.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:37.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:37.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.694649383000002, 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:37.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:37.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:37.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.693693773, 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:37.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:37.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:37.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.693037074999999, 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:37.377 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:37.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.692505318999999, 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:37.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:37.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:37.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.691691659, 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:37.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:37.390 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.691219755999999, 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:37.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.690190124000001, 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:37.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.689567475, 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:37.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:37.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.683981736, 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:37.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.683021466, 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.682260631999998, 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.416 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:37.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:37.418 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:37.415+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:37.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:37:37.519 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:37.518+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:37.586 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:37.586 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:37.587 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:37.587 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:37.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:37.620 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:37.623+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:37.721 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:37.718+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:37.763 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:37.763 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:37.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:37.822 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:37.823+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:37.922 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:37.927+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:37.937 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.676994289, 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:37:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.658906363, 2024-09-21T23:37:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:37:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:37.947 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:37.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:38.023 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:38.027+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:38.069 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:38.069 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:38.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:38.130 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:38.130+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:38.202 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:38.202 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:38.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:38.226 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.231+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.327 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.331+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.379 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.111606369, 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:38.388 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:38.393 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:38.425 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:37:38.442 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.447+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.529 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.527+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.530 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:38.530 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:38.530 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:38.530 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:38.544 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:38.630 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:38.630+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:38.711 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:38.712 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:38.712 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:38.712 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:38.712 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:38.727 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:38.774 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.775+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:38.832 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:38.831+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.623801610000001, 2024-09-21T23:37:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.854616154999999, 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.867 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.854484631999998, 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:38.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:38.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.854424723000001, 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:38.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:38.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:38.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.753597447000001, 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:38.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:38.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:38.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.889 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.753481957999998, 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:38.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:38.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.587210245000001, 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.896 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:38.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:38.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.587163335, 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.587121949, 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:38.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:38.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.586945076999999, 2024-09-21T23:37:38.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:38.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:38.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:38.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.586897381, 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:38.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.928 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.586537196999998, 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.929 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:38.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:38.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.586409240999998, 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:38.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:38.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.586371965000001, 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:38.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:38.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:38.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.222612560000002, 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:38.949 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:38.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:38.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.956 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.2225778, 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:38.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:38.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.222481874, 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:38.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:38.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.607760423999999, 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:38.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:38.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.975 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.607701568, 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:38.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.607588374999999, 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:38.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:38.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:38.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:38.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:38.989 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:38.990 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:38.991 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:38.991 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:38.992 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:38.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:38.992 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:38.993 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:38.993 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:38.990+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:38.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:38.993 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:39.014 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:39.014 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:39.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:39.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:39.091 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:39.095+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:39.167 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.167 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.182 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:39.192 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:39.190+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:39.294 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:39.294+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:39.353 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.353 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:39.394 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:39.394+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:39.493 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.493 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.495 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:39.494+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:39.507 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:39.595 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:39.599+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:39.625 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.625 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.640 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:39.696 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:39.694+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:39.797 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:39.794+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:39.803 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.803 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.817 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:39.898 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:39.899+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:39.940 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:39.940 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:39.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:39.999 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:39.998+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:40.084 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.085 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:40.100 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:40.103+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:40.201 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:40.203+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:40.264 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.265 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:40.302 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:40.307+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:40.402 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.402 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.402 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:40.407+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:40.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:40.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:37:40.506 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:40.506+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:40.540 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.540 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:40.605 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:40.607+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:40.706 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:40.706+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:40.710 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.710 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:40.806 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:40.811+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:40.811 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:37:40.877 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:40.877 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:40.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:40.907 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:40.911+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:41.007 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:41.007 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:41.008 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:41.011+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:41.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:41.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:41.110 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:41.110+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:41.142 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:41.142 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:41.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:41.203 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:41.204 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:41.219 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:41.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:41.220 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:41.218+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:41.236 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:41.236 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:41.256 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:41.257 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:41.277 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:41.277 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:41.298 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:41.298 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:41.307 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:41.307 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.040729168999999, 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:41.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:41.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:41.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:41.312 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:41.314 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:41.315+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:41.321 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:41.415 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:41.419+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:41.437 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:41.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:37:41.452 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:41.452 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:41.453 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:41.453 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:41.467 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:41.515 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:41.514+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:41.616 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:41.619+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:41.635 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:41.635 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:41.635 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:41.635 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:41.635 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:41.649 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:41.734 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:41.739+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.585793471999999, 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:41.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.816608017, 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:41.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:41.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.816476494, 2024-09-21T23:37:41.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:41.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:41.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.816416584999999, 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:41.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.840 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.715589308999999, 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:41.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:41.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.71547382, 2024-09-21T23:37:41.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:41.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.849 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:41.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.549202106999999, 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:41.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.549155197000001, 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.861 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.866 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.549113811000002, 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.867 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:41.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.548936939000001, 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.873 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:41.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:41.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.548889243000001, 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:41.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:41.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.548529059, 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.885 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:41.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.548401103, 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.891 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:41.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:41.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.897 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.548363826999999, 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:41.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.184604422, 2024-09-21T23:37:41.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:41.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:41.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:41.909 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.184569662000001, 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:41.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.184473736000001, 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:41.915 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.921 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.569752286, 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.922 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:41.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:41.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:41.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:41.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:41.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:41.932 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.569693430000001, 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:41.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.569580237, 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:41.938 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:41.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:41.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:41.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:41.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:41.944 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:41.946 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:37:41.946 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:41.946 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:41.947+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:41.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:37:42.047 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:42.046+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:42.120 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.120 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:42.121 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:42.121 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.135 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:42.178 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:42.183+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:42.249 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:42.246+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:42.251 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:42.252 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.266 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:42.350 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:42.350+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:42.431 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.432 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:42.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:42.432 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:37:42.432 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.445 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:42.490 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:42.495+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:42.552 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:42.550+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:42.566 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.566 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:42.566 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:42.566 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.581 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:42.653 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:42.650+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:42.743 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.743 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:42.743 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:42.743 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:42.743 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.754 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:42.759+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:42.757 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:42.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:37:42.836 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:37:42.855 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:42.859+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:42.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:37:42.925 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:42.925 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:42.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:42.925 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:42.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.662081247, 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:42.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:42.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:42.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.655324226000001, 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:42.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.650000566999999, 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:42.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:42.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.649132956999999, 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.941 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:42.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.664934899999999, 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:42.945 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:42.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:42.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.662233670999999, 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:42.951 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:42.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:42.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.65049776, 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:42.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:42.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:42.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.643745129999999, 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:42.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:42.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:42.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:42.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:42.966 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:42.967 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:42.971+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:42.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:43.068 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:43.066+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:43.080 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:43.081 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:43.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:43.169 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:43.171+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:43.250 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:43.250 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:43.250 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:43.250 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:43.250 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:43.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:43.269 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:43.271+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:43.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:43.370 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:43.375+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.407885721, 2024-09-21T23:37:43.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:43.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:43.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.372730282999999, 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:43.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.359259553000001, 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:43.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.335585456, 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:43.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:43.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.272258494999999, 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:43.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.100088500999998, 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:43.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:43.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.749449206000001, 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:43.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:43.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.748741418000002, 2024-09-21T23:37:43.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:43.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:43.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.748193909000001, 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:43.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.747541188, 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:43.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:43.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.746585578000001, 2024-09-21T23:37:43.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:43.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:43.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.745928880000001, 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:43.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:43.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:43.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.745397124, 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:43.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:43.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.744583464000002, 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:43.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:43.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.744111561, 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:43.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.743081928999999, 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:43.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:43.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:43.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.742459279999999, 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:43.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.736873541000001, 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:43.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:43.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:43.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.735913271000001, 2024-09-21T23:37:43.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:43.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:43.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.735152437, 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:43.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:43.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:43.470 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:43.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:43.472 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:43.472 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:43.472 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:43.473 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:43.473 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:43.475+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:43.488 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:43.488 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:43.508 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:43.509 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:43.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:43.529 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:43.549 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:43.585 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:43.585 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:43.586 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:43.586+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:43.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:43.672 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:43.675+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:43.756 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:43.756 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:43.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:43.774 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:43.770+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:43.852 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:37:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:37:43.874 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:43.879+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:43.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:37:43.895 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:43.895 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:43.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:43.994 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:43.994+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:44.065 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.065 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.076 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:44.074+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:44.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:44.178 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:44.179+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:44.242 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.243 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:44.302 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:44.302+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:44.370 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.370 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.379 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:44.378+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:44.385 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:44.480 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:44.478+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:44.516 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.516 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.531 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:44.581 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:44.583+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:44.648 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.648 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.662 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:44.706 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:44.711+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:44.784 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:44.787+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:44.830 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:44.830 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:44.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:44.847 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:37:44.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:37:44.886 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:44.891+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:44.986 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:44.987+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:45.005 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:45.005 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:45.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:45.106 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:45.111+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:45.187 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:45.186+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:45.187 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:45.187 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:45.201 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:45.288 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:45.286+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:45.366 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:45.367 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:45.380 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:45.389 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:45.386+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:45.490 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:45.486+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:45.504 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:45.504 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:45.518 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:45.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:45.591 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:45.595+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:45.636 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:45.636 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:45.650 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:45.692 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:45.690+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:45.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:45.715 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:45.730 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:45.731 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:45.746 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:45.746 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:45.765 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:45.765 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:45.785 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:45.785 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:45.792 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:45.790+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:45.806 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:45.806 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.553122674000001, 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.546365652999999, 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.541041994, 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.540174384, 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.555976327, 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.553275098, 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.541539187000001, 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.534786557, 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:45.862 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:45.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:45.864 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:37:45.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:37:45.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:37:45.893 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:45.895+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:45.979 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:45.979 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:45.979 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:45.979 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:45.994 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:45.994+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:45.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:46.095 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:46.099+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:46.157 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:46.158 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:46.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:46.218 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:46.218+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:46.285 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:46.285 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:46.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:46.285 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.314838532000003, 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:46.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.279683093999999, 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:46.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.266212363999998, 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:46.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:46.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:46.299 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.242538267, 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:46.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.179211305999999, 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:46.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.007041311999998, 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:46.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.656402016999998, 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:46.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.655694228999998, 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:46.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:46.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:46.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.655146719999998, 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:46.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:46.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:46.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.654493999000003, 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.653538388999998, 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:46.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.652881690999997, 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:46.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:46.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.652349934999997, 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:46.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.651536274999998, 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:46.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.651064372, 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:46.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:46.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.650034740000002, 2024-09-21T23:37:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.649412091000002, 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.643826351999998, 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.642866081999998, 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.642105248, 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.375 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:46.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:46.377 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:46.374+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:46.478 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:46.478+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:46.539 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:46.539 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:46.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:46.594 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:46.594+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:46.680 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:46.678+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:46.715 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:46.715 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:46.729 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:46.814 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:46.814+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:46.861 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:37:46.861 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:37:46.882 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:46.882+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:46.885 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:37:46.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:46.904 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.645076997, 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:37:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:37:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.626989071000001, 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:37:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:37:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:37:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:46.914 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:46.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:46.983 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:46.987+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:47.075 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:47.076 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:47.084 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:47.082+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:47.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:47.185 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:47.182+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:47.191 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-21T23:37:46.958849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":190180,"kb_used_data":3028,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566040860,"statfs":{"total":579820584960,"available":579625840640,"internally_reserved":0,"allocated":3100672,"data_stored":1849594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000996"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855040+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:44.855040+0000","last_peered":"2024-09-21T23:37:44.855040+0000","last_clean":"2024-09-21T23:37:44.855040+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:44.855040+0000","last_undegraded":"2024-09-21T23:37:44.855040+0000","last_fullsized":"2024-09-21T23:37:44.855040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-21T23:37:43.852160+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:43.852160+0000","last_peered":"2024-09-21T23:37:43.852160+0000","last_clean":"2024-09-21T23:37:43.852160+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:43.852160+0000","last_undegraded":"2024-09-21T23:37:43.852160+0000","last_fullsized":"2024-09-21T23:37:43.852160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855037+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:44.855037+0000","last_peered":"2024-09-21T23:37:44.855037+0000","last_clean":"2024-09-21T23:37:44.855037+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:44.855037+0000","last_undegraded":"2024-09-21T23:37:44.855037+0000","last_fullsized":"2024-09-21T23:37:44.855037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-21T23:37:43.852055+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:43.852055+0000","last_peered":"2024-09-21T23:37:43.852055+0000","last_clean":"2024-09-21T23:37:43.852055+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:43.852055+0000","last_undegraded":"2024-09-21T23:37:43.852055+0000","last_fullsized":"2024-09-21T23:37:43.852055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:45.051579+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:45.051579+0000","last_peered":"2024-09-21T23:37:45.051579+0000","last_clean":"2024-09-21T23:37:45.051579+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:45.051579+0000","last_undegraded":"2024-09-21T23:37:45.051579+0000","last_fullsized":"2024-09-21T23:37:45.051579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.851974+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:44.851974+0000","last_peered":"2024-09-21T23:37:44.851974+0000","last_clean":"2024-09-21T23:37:44.851974+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:44.851974+0000","last_undegraded":"2024-09-21T23:37:44.851974+0000","last_fullsized":"2024-09-21T23:37:44.851974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855060+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:44.855060+0000","last_peered":"2024-09-21T23:37:44.855060+0000","last_clean":"2024-09-21T23:37:44.855060+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:44.855060+0000","last_undegraded":"2024-09-21T23:37:44.855060+0000","last_fullsized":"2024-09-21T23:37:44.855060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:45.051581+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:45.051581+0000","last_peered":"2024-09-21T23:37:45.051581+0000","last_clean":"2024-09-21T23:37:45.051581+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:45.051581+0000","last_undegraded":"2024-09-21T23:37:45.051581+0000","last_fullsized":"2024-09-21T23:37:45.051581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855114+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:44.855114+0000","last_peered":"2024-09-21T23:37:44.855114+0000","last_clean":"2024-09-21T23:37:44.855114+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:44.855114+0000","last_undegraded":"2024-09-21T23:37:44.855114+0000","last_fullsized":"2024-09-21T23:37:44.855114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607563,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":327680,"data_stored":113778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607564,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35672,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336168,"statfs":{"total":96636764160,"available":96600236032,"internally_reserved":0,"allocated":323584,"data_stored":113759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607563,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335592,"statfs":{"total":96636764160,"available":96599646208,"internally_reserved":0,"allocated":917504,"data_stored":704127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607563,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":104603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607562,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":113778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607563,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":905216,"data_stored":699549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:47.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:47.243 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:47.243 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:47.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:47.302 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:47.302+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:47.387 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:47.391+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:47.419 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.152630033000001, 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:47.424 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:47.433 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:47.487 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:47.486+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:47.516 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:47.533 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-21T23:37:46.958849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":190180,"kb_used_data":3028,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566040860,"statfs":{"total":579820584960,"available":579625840640,"internally_reserved":0,"allocated":3100672,"data_stored":1849594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000996"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855040+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:44.855040+0000","last_peered":"2024-09-21T23:37:44.855040+0000","last_clean":"2024-09-21T23:37:44.855040+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:44.855040+0000","last_undegraded":"2024-09-21T23:37:44.855040+0000","last_fullsized":"2024-09-21T23:37:44.855040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-21T23:37:43.852160+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:43.852160+0000","last_peered":"2024-09-21T23:37:43.852160+0000","last_clean":"2024-09-21T23:37:43.852160+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:43.852160+0000","last_undegraded":"2024-09-21T23:37:43.852160+0000","last_fullsized":"2024-09-21T23:37:43.852160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855037+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:44.855037+0000","last_peered":"2024-09-21T23:37:44.855037+0000","last_clean":"2024-09-21T23:37:44.855037+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:44.855037+0000","last_undegraded":"2024-09-21T23:37:44.855037+0000","last_fullsized":"2024-09-21T23:37:44.855037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-21T23:37:43.852055+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:43.852055+0000","last_peered":"2024-09-21T23:37:43.852055+0000","last_clean":"2024-09-21T23:37:43.852055+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:43.852055+0000","last_undegraded":"2024-09-21T23:37:43.852055+0000","last_fullsized":"2024-09-21T23:37:43.852055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:45.051579+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:45.051579+0000","last_peered":"2024-09-21T23:37:45.051579+0000","last_clean":"2024-09-21T23:37:45.051579+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:45.051579+0000","last_undegraded":"2024-09-21T23:37:45.051579+0000","last_fullsized":"2024-09-21T23:37:45.051579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.851974+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:44.851974+0000","last_peered":"2024-09-21T23:37:44.851974+0000","last_clean":"2024-09-21T23:37:44.851974+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:44.851974+0000","last_undegraded":"2024-09-21T23:37:44.851974+0000","last_fullsized":"2024-09-21T23:37:44.851974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855060+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:44.855060+0000","last_peered":"2024-09-21T23:37:44.855060+0000","last_clean":"2024-09-21T23:37:44.855060+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:44.855060+0000","last_undegraded":"2024-09-21T23:37:44.855060+0000","last_fullsized":"2024-09-21T23:37:44.855060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:45.051581+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:45.051581+0000","last_peered":"2024-09-21T23:37:45.051581+0000","last_clean":"2024-09-21T23:37:45.051581+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:45.051581+0000","last_undegraded":"2024-09-21T23:37:45.051581+0000","last_fullsized":"2024-09-21T23:37:45.051581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T23:37:44.855114+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:44.855114+0000","last_peered":"2024-09-21T23:37:44.855114+0000","last_clean":"2024-09-21T23:37:44.855114+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:44.855114+0000","last_undegraded":"2024-09-21T23:37:44.855114+0000","last_fullsized":"2024-09-21T23:37:44.855114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607563,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":327680,"data_stored":113778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607564,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35672,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336168,"statfs":{"total":96636764160,"available":96600236032,"internally_reserved":0,"allocated":323584,"data_stored":113759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607563,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335592,"statfs":{"total":96636764160,"available":96599646208,"internally_reserved":0,"allocated":917504,"data_stored":704127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607563,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":104603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607562,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":113778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607563,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":905216,"data_stored":699549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:47.534 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:37:47.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:47.534 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T23:37:47.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T23:37:47.568 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:47.568 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:47.568 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:47.568 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:47.582 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:47.626 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:47.631+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:47.690 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:47.690+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:47.699 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:47.699 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:47.699 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:47.700 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:47.700 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:47.715 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:47.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T23:37:47.802 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:47.807+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:47.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.655268089000003, 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.883 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:47.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:47.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.886082633999997, 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:47.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:47.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.885951110999997, 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:47.896 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:47.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.885891202000003, 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:47.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:47.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.785063925999999, 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:47.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.784948436999997, 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.914 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:47.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:47.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618676723999997, 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:47.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:47.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:47.925 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618629814000002, 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:47.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618588428000002, 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:47.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618411555999998, 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:47.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:47.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618363860000002, 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:47.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:47.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.618003676000001, 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:47.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.617875720000001, 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:47.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:47.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.617838444, 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.961 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:47.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.966 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.254079038999997, 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:47.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.254044278999999, 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:47.972 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:47.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:47.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:47.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.253948352999998, 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:47.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:47.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.639226903000001, 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.639168046999998, 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.990 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:47.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:47.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.639054854000001, 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.996 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:47.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:47.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:47.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:48.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:48.001 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:48.002 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:48.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T23:37:48.004 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:48.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:48.005 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:48.007+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:48.005 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:48.018 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:48.018 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:48.033 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:48.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:48.055 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:48.055 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:48.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:48.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:48.092 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:48.092 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:48.103 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:48.107+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:48.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T23:37:48.180 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:48.180 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:48.194 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:48.242 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:48.242+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:48.305 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:48.302+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:48.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T23:37:48.361 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:48.361 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:48.375 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:48.422 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:48.427+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:48.492 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:48.492 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:48.506 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:48.506+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:48.507 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:48.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T23:37:48.607 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:48.611+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:48.679 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:48.679 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:48.694 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:48.734 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:48.739+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:48.809 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:48.811+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:48.858 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:48.858 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:48.872 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:48.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:37:48.914 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:48.919+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:49.011 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:49.015+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:49.037 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.037 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:49.112 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:49.115+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:49.163 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.163 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:49.212 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:49.215+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:49.291 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.292 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:49.313 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:49.315+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:49.414 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:49.419+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:49.426 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.426 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:49.515 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:49.519+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:49.560 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.560 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:49.618 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:49.618+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:49.698 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.698 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:49.754 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:49.754+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:49.817 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:49.819+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:49.873 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:49.873 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:49.884 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:37:49.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:49.918 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:49.919+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:50.002 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:50.003 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:50.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:50.018 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:50.023+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:50.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:50.119 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:50.123+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:50.125 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:50.125 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:50.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:50.230 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:50.230+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:50.270 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:50.270 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:50.289 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:50.289 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:50.306 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:50.306 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.050042685000001, 2024-09-21T23:37:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:50.327 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:50.328 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:50.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:50.329 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:50.331+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:50.331 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:50.347 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:50.347 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:50.366 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:50.367 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:50.430 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:50.430+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:50.457 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:50.458 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:50.458 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:50.458 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:50.472 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:50.530 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:50.535+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:50.596 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:50.748 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:50.748 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:50.748 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:50.748 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:50.750 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:50.750 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:50.750+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:50.851 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:50.850+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:50.925 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:50.925 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:50.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:50.925 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.695893708, 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:50.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:50.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:50.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:50.932 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.926708253000001, 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:50.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:50.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:50.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.926576730000001, 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.938 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:50.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.926516821, 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.944 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:50.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:50.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:50.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:50.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.825689545000003, 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:50.949 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:50.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:50.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:50.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.825574056000001, 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:50.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.659302343, 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:50.961 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:50.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:50.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:50.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.659255432999998, 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.659214046999999, 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:50.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:50.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.659037175000002, 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:50.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:50.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.658989478999999, 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.985 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:50.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:50.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.658629294999997, 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:50.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:50.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:50.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:50.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.658501338999997, 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:50.997 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:50.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:50.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:51.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:51.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:51.003 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.658464062999997, 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:51.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:51.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.009 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.294704658000001, 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:51.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:51.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.018 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.294669898000002, 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.019 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:51.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.294573972000002, 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.028 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:51.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.679852521999997, 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.035 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:51.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:51.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:51.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.679793666000002, 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:51.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.679680472999998, 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:51.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.052 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.053 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.054 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:51.054 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:51.059+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:51.155 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:51.154+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:51.227 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.227 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:51.227 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:51.228 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:51.255 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:51.254+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:51.356 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:51.359+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:51.360 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.360 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:51.360 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:51.360 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:51.361 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.374 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:51.466 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:51.471+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:51.542 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:37:51.542 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:51.557 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:51.558+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:51.659 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:51.658+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:51.718 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.718 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:51.719 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:51.719 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.732 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:51.760 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:51.758+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:51.848 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.849 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:51.849 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:51.849 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:51.849 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:51.860 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:51.858+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:51.862 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:51.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:37:51.961 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:51.958+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:51.983 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.720597982999998, 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.713840961999999, 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.708517303000001, 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.707649693, 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.723451636, 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.720750406999997, 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.709014496000002, 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:52.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:52.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.702261866000001, 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:52.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:52.023 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.024 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.024 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.024 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:52.024 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:52.024 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:52.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:52.062 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:52.067+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:52.142 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:52.142 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:52.142 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:52.142 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:52.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:52.206 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:52.206+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:52.264 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:52.267+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:52.267 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:52.267 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:52.267 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:52.268 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:52.268 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:52.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:52.364 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:52.367+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:52.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:52.445 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:52.445 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:52.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:52.445 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.474730682000001, 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:52.451 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.439575243999997, 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:52.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:52.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:52.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.426104514000002, 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:52.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.402430416999998, 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.339103455999997, 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:52.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.166933462000003, 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:52.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.816294167000002, 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:52.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.815586379000003, 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:52.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:52.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:52.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:52.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.815038870000002, 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:52.485 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:52.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:52.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.814386149000001, 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:52.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:52.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.813430539000002, 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.812773841000002, 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.812242085000001, 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.811428425000003, 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.810956521999998, 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.80992689, 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.809304241, 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.803718502000002, 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.802758232000002, 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.801997397999997, 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:52.544 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:52.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:52.546 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:52.547 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:52.547 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:52.548 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:52.548 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:52.548 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:52.542+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:52.561 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:52.561 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:52.580 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:52.580 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:52.600 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:52.600 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:52.647 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:52.651+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:52.679 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:52.679 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:52.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:52.747 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:52.746+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:52.811 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:52.811 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:52.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:52.848 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:52.851+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:52.908 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:37:52.970 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:52.970+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:52.988 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:52.988 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:53.051 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:53.055+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:53.151 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:53.150+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:53.170 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.170 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:53.270 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:53.270+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:53.350 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.350 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.353 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:53.350+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:53.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:53.454 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:53.459+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:53.529 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.529 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.544 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:53.555 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:53.554+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:53.656 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:53.654+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:53.676 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.676 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.691 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:53.744 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:53.744 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:53.745 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T23:37:53.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:37:53.794 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:53.794+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:53.812 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.812 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.827 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:53.874 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:53.879+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:53.949 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:53.949 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:53.958 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:53.958+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:53.963 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:54.054 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 36, 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66973394-6d88-4f14-bf11-7ccd7c9cf4a7", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-21T23:36:58.086363+0000", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-21T23:37:52.895513+0000", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-21T23:37:09.181667+0000", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-21T23:36:59.825771+0000", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-21T23:37:54.055 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-21T23:37:54.056 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 6, 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "squid", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:11.018666+0000", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-21T23:37:54.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-21T23:37:54.058 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "16", 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:37:54.059 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:37:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:37:54.061 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-21T23:37:54.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:37:54.063 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:15.320283+0000", 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:37:54.064 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:37:54.065 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 2, 2024-09-21T23:37:54.066 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 20, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:37:54.067 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:37:54.068 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.25, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.25, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.25, 2024-09-21T23:37:54.069 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.25, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "638e3e48-1763-4ec9-8c06-634e0b3b793b", 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.070 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/915397992", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/915397992", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/915397992", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/915397992", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "02967526-d5c5-4f68-8e25-343311dbaec4", 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:37:54.076 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:37:54.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/3900686792", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/3900686792", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/3900686792", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/3900686792", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "4f770ac1-d062-4934-81d5-7f980b3830a1", 2024-09-21T23:37:54.081 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.085 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/2564679849", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/2564679849", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6823/2564679849", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/2564679849", 2024-09-21T23:37:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "90cc3313-3845-48d0-839d-8fea4c52bfde", 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.087 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6808", 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.088 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6809", 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6810", 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6811", 2024-09-21T23:37:54.089 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6814", 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.090 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6815", 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.091 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6812", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6813", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6809/1410364236", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6811/1410364236", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6815/1410364236", 2024-09-21T23:37:54.092 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6813/1410364236", 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "42b8e641-af88-4d41-bc5f-a14020f205ce", 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.093 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6800", 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.094 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6801", 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6802", 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.095 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6803", 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6806", 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.096 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6807", 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6804", 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6805", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6801/1572346429", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6803/1572346429", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6807/1572346429", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6805/1572346429", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.098 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9", 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:37:54.099 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6816", 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6817", 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6818", 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6819", 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.101 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6822", 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6823", 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6820", 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6821", 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:37:54.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6817/466572374", 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6819/466572374", 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6823/466572374", 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6821/466572374", 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:37:54.104 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.810277+0000", 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.105 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.686068+0000", 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.106 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.949768+0000", 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.773303+0000", 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:37:54.107 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.009610+0000", 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:37:54.108 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.059224+0000", 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-21T23:37:54.109 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-21T23:37:54.110 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-21T23:37:54.111 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:37:54.111 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:37:54.111 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-21T23:37:54.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:54.111 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:54.112 INFO:tasks.thrashosds.thrasher:j is {'epoch': 36, 'fsid': '66973394-6d88-4f14-bf11-7ccd7c9cf4a7', 'created': '2024-09-21T23:36:58.086363+0000', 'modified': '2024-09-21T23:37:52.895513+0000', 'last_up_change': '2024-09-21T23:37:09.181667+0000', 'last_in_change': '2024-09-21T23:36:59.825771+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:37:11.018666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:37:15.320283+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '638e3e48-1763-4ec9-8c06-634e0b3b793b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 915397992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 915397992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 915397992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 915397992}]}, 'public_addr': '172.21.15.79:6801/915397992', 'cluster_addr': '172.21.15.79:6803/915397992', 'heartbeat_back_addr': '172.21.15.79:6807/915397992', 'heartbeat_front_addr': '172.21.15.79:6805/915397992', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02967526-d5c5-4f68-8e25-343311dbaec4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 3900686792}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 3900686792}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 3900686792}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 3900686792}]}, 'public_addr': '172.21.15.79:6809/3900686792', 'cluster_addr': '172.21.15.79:6811/3900686792', 'heartbeat_back_addr': '172.21.15.79:6815/3900686792', 'heartbeat_front_addr': '172.21.15.79:6813/3900686792', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4f770ac1-d062-4934-81d5-7f980b3830a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2564679849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 2564679849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 2564679849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 2564679849}]}, 'public_addr': '172.21.15.79:6817/2564679849', 'cluster_addr': '172.21.15.79:6819/2564679849', 'heartbeat_back_addr': '172.21.15.79:6823/2564679849', 'heartbeat_front_addr': '172.21.15.79:6821/2564679849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '90cc3313-3845-48d0-839d-8fea4c52bfde', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 1410364236}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 1410364236}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 1410364236}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 1410364236}]}, 'public_addr': '172.21.15.118:6809/1410364236', 'cluster_addr': '172.21.15.118:6811/1410364236', 'heartbeat_back_addr': '172.21.15.118:6815/1410364236', 'heartbeat_front_addr': '172.21.15.118:6813/1410364236', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '42b8e641-af88-4d41-bc5f-a14020f205ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 1572346429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 1572346429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 1572346429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 1572346429}]}, 'public_addr': '172.21.15.118:6801/1572346429', 'cluster_addr': '172.21.15.118:6803/1572346429', 'heartbeat_back_addr': '172.21.15.118:6807/1572346429', 'heartbeat_front_addr': '172.21.15.118:6805/1572346429', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6817', 'nonce': 466572374}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6818', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 466572374}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6822', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 466572374}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6821', 'nonce': 466572374}]}, 'public_addr': '172.21.15.118:6817/466572374', 'cluster_addr': '172.21.15.118:6819/466572374', 'heartbeat_back_addr': '172.21.15.118:6823/466572374', 'heartbeat_front_addr': '172.21.15.118:6821/466572374', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.810277+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.686068+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.949768+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.773303+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.009610+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.059224+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:37:54.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:37:54.113 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:54.110+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:54.127 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:54.127 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:54.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:54.213 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:54.210+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:54.262 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:54.262 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:54.276 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:54.313 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:54.314+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:54.392 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:54.392 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:54.408 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:54.415 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:54.414+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:54.446 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:37:54.447 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:37:54.462 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-21T23:37:52.960047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":190448,"kb_used_data":3280,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566040592,"statfs":{"total":579820584960,"available":579625566208,"internally_reserved":0,"allocated":3358720,"data_stored":1945732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.092296"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.896251+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:49.896251+0000","last_peered":"2024-09-21T23:37:49.896251+0000","last_clean":"2024-09-21T23:37:49.896251+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:49.896251+0000","last_undegraded":"2024-09-21T23:37:49.896251+0000","last_fullsized":"2024-09-21T23:37:49.896251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.891169+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:49.891169+0000","last_peered":"2024-09-21T23:37:49.891169+0000","last_clean":"2024-09-21T23:37:49.891169+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:49.891169+0000","last_undegraded":"2024-09-21T23:37:49.891169+0000","last_fullsized":"2024-09-21T23:37:49.891169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.896226+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:49.896226+0000","last_peered":"2024-09-21T23:37:49.896226+0000","last_clean":"2024-09-21T23:37:49.896226+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:49.896226+0000","last_undegraded":"2024-09-21T23:37:49.896226+0000","last_fullsized":"2024-09-21T23:37:49.896226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.890981+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:49.890981+0000","last_peered":"2024-09-21T23:37:49.890981+0000","last_clean":"2024-09-21T23:37:49.890981+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:49.890981+0000","last_undegraded":"2024-09-21T23:37:49.890981+0000","last_fullsized":"2024-09-21T23:37:49.890981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":49,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T23:37:47.282145+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:47.282145+0000","last_peered":"2024-09-21T23:37:47.282145+0000","last_clean":"2024-09-21T23:37:47.282145+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:47.282145+0000","last_undegraded":"2024-09-21T23:37:47.282145+0000","last_fullsized":"2024-09-21T23:37:47.282145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T23:37:46.864764+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:46.864764+0000","last_peered":"2024-09-21T23:37:46.864764+0000","last_clean":"2024-09-21T23:37:46.864764+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:46.864764+0000","last_undegraded":"2024-09-21T23:37:46.864764+0000","last_fullsized":"2024-09-21T23:37:46.864764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.896077+0000","last_change":"2024-09-21T23:37:19.278536+0000","last_active":"2024-09-21T23:37:49.896077+0000","last_peered":"2024-09-21T23:37:49.896077+0000","last_clean":"2024-09-21T23:37:49.896077+0000","last_became_active":"2024-09-21T23:37:17.265579+0000","last_became_peered":"2024-09-21T23:37:17.265579+0000","last_unstale":"2024-09-21T23:37:49.896077+0000","last_undegraded":"2024-09-21T23:37:49.896077+0000","last_fullsized":"2024-09-21T23:37:49.896077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:17.640442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001886106,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":48,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T23:37:47.282198+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:47.282198+0000","last_peered":"2024-09-21T23:37:47.282198+0000","last_clean":"2024-09-21T23:37:47.282198+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:47.282198+0000","last_undegraded":"2024-09-21T23:37:47.282198+0000","last_fullsized":"2024-09-21T23:37:47.282198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T23:37:49.896182+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:49.896182+0000","last_peered":"2024-09-21T23:37:49.896182+0000","last_clean":"2024-09-21T23:37:49.896182+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:49.896182+0000","last_undegraded":"2024-09-21T23:37:49.896182+0000","last_fullsized":"2024-09-21T23:37:49.896182+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607564,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":364544,"data_stored":127512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35708,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":360448,"data_stored":127493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607564,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":954368,"data_stored":717861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607564,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":360448,"data_stored":127493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607563,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":127512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607564,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":954368,"data_stored":717861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:37:54.463 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [5, 2] 2024-09-21T23:37:54.463 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.0', '5', '2'] 2024-09-21T23:37:54.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.0 5 2 2024-09-21T23:37:54.526 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:54.526 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:54.542 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:54.550 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:54.550+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:54.616 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:54.619+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:54.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:54.664 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:54.664 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:54.678 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:54.717 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:54.714+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:54.753 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:54.753 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:54.769 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:54.770 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:54.787 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:54.787 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:54.806 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:54.807 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:54.818 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:54.823+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:54.825 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:54.825 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:54.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:54.845 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:54.850 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.587703634, 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:37:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:37:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.580946613000002, 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:37:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:37:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.575622953999996, 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:37:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.574755344000003, 2024-09-21T23:37:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:37:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.590557287000003, 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:37:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:37:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:37:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.587856058, 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:37:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.576120146999997, 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:37:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:37:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:37:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:37:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:37:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.569367517000003, 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:54.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:54.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:54.897 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:54.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:54.913 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.0 pg_upmap mapping to [5,2] 2024-09-21T23:37:54.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:37:54.942 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:54.947+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:55.020 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:55.023+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:55.036 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:55.036 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:55.120 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:55.123+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:55.156 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:55.156 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:55.221 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:55.218+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:55.276 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.306338465000003, 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:37:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:37:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:37:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.271183026999999, 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:37:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:37:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.257712296999998, 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:37:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.234038200000001, 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:37:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:37:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.170711238999999, 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:37:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:37:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.998541244999998, 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:37:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:37:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:37:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.647901949999998, 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:37:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:37:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.647194161999998, 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:37:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.646646652999998, 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:37:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:37:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:37:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:37:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.645993932000003, 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.645038321999998, 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:37:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.644381623999998, 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:37:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:37:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:37:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.643849867999997, 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:37:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:37:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.643036207999998, 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:37:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:37:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.642564305000001, 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:37:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:37:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:37:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.641534673000002, 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:37:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.640912024000002, 2024-09-21T23:37:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.635326284999998, 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:37:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:37:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.634366014999998, 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:37:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.633605181, 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:37:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:37:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:37:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.374 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:55.376 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:55.374+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:55.477 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:55.479+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:55.539 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:55.598 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:55.598+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:55.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:55.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.678 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:55.683+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:55.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:55.779 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:55.778+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.591082067000002, 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:37:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:37:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.572994141000002, 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:37:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:37:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:55.860 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:55.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:55.881 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:55.883+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:55.921 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:37:55.981 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:55.983+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:56.022 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:37:56.023 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:56.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:56.082 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:56.082+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:56.153 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:37:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:37:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:37:56.154 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:56.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:56.210 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:56.210+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:56.283 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:56.287+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.063954072000001, 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:37:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:37:56.335 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:37:56.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:56.384 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:56.382+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:56.476 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:56.476 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:56.476 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:56.476 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:56.485 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:56.487+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:56.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:56.586 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:56.591+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:56.656 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:56.656 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:56.656 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:56.656 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:56.670 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:56.714 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:56.719+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:56.787 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:56.791+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.611363347999998, 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.842177892999999, 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.842046369999998, 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.841986460999998, 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.741159185000001, 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.741043695999998, 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574771982999998, 2024-09-21T23:37:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:56.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:56.894 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574725073000003, 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:56.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:56.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:56.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:56.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:56.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574683686999997, 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:56.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:56.908 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574506814999999, 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:56.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:56.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:56.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574459118999997, 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:56.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.574098935000002, 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.922 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:56.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.928 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.573970979000002, 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.929 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:56.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:56.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.573933703000002, 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:56.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.940 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.210174297999998, 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:56.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.210139538, 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:56.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:56.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:56.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.210043612, 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:56.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.595322162000002, 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:56.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.966 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.595263306, 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:56.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:56.972 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.595150113000003, 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:56.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:56.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:56.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:56.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:56.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:56.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:56.979 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:56.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:37:56.981 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:56.978+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:56.982 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:57.082 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:57.082+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:57.133 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:57.133 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:57.149 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:57.150 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:57.158 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:57.158 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:57.158 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:57.158 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:57.165 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:57.166 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:57.172 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:37:57.182 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:57.182+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:57.184 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:57.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:57.202 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:57.202 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:57.219 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:57.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:57.283 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:57.287+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:57.296 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:57.296 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:57.310 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:37:57.384 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:57.382+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:57.475 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:57.475 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:57.485 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:57.487+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:57.489 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:37:57.586 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:57.586+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:57.654 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:57.654 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:57.669 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:37:57.714 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:57.719+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:57.787 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:57.791+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:57.789 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:57.789 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:57.804 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:37:57.894 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:57.899+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:57.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:37:57.973 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:57.974 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:57.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:37:58.030 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:58.035+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:58.091 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:58.090+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:58.121 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.121 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:37:58.226 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:58.226+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:58.292 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:58.295+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:58.293 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.293 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:37:58.393 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:58.395+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:58.472 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.472 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:37:58.493 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:58.495+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:58.594 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:58.599+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:58.607 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.607 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:37:58.695 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:58.699+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:58.741 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.741 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:37:58.795 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:58.799+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:58.875 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:58.875 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:58.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:37:58.937 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:37:58.938 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:37:58.938+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:37:59.002 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:37:59.002+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:37:59.012 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:59.012 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:59.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:37:59.099 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:59.103+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:59.191 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:59.192 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:59.199 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:37:59.203+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:37:59.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:37:59.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:37:59.300 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:59.303+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:59.328 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:37:59.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:37:59.341 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:37:59.383 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:37:59.383 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:37:59.398 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:37:59.398 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:37:59.400 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:59.398+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:59.414 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:37:59.414 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:37:59.433 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:37:59.433 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:37:59.453 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:37:59.453 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:37:59.473 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:37:59.473 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:37:59.476 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:37:59.476 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:59.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:37:59.534 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:37:59.534+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:37:59.602 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:37:59.607+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:37:59.612 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:59.612 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:37:59.612 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:37:59.612 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:37:59.612 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:59.626 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:37:59.718 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:59.723+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.565501906999998, 2024-09-21T23:37:59.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:59.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:37:59.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.796316451999999, 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:37:59.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:37:59.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.796184928999999, 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:37:59.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:37:59.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.796125019999998, 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:37:59.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:37:59.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.695297744000001, 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:37:59.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:37:59.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.695182254999999, 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:37:59.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528910541999998, 2024-09-21T23:37:59.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:59.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:37:59.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528863631999997, 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:37:59.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:37:59.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528822245999997, 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:37:59.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:37:59.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:37:59.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:37:59.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528645374, 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.854 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528597677999997, 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:37:59.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528237494000003, 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:37:59.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:59.871 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528109538000002, 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:59.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:37:59.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:37:59.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.528072262000002, 2024-09-21T23:37:59.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:37:59.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:37:59.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.164312856999999, 2024-09-21T23:37:59.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:37:59.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:37:59.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.164278097, 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:37:59.889 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:37:59.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:37:59.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.164182171, 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.895 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:37:59.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:37:59.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.549460721000003, 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.901 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:37:59.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.549401865, 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.907 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:59.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:37:59.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:37:59.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.549288672000003, 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:37:59.913 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:37:59.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:37:59.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:37:59.919 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:37:59.921 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:37:59.921 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:37:59.923+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:37:59.934 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:37:59.934 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:37:59.935 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:37:59.935 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.870786 2024-09-21T23:37:59.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.8707862309245075 2024-09-21T23:38:00.021 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:00.018+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:00.092 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.092 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:00.092 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:00.092 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.106 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:00.150 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:00.155+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:00.223 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:00.222+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:00.224 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.224 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:00.224 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:00.224 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:00.224 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.238 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:00.330 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:00.335+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:00.404 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.404 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:00.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:00.404 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:38:00.405 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.418 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:00.462 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:00.467+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:00.527 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:00.526+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:00.583 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.584 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:00.584 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:00.584 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.599 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:00.642 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:00.647+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:00.717 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.717 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:00.717 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:00.717 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:00.718 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.728 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:00.731+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:00.731 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:00.829 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:00.826+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:00.901 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:00.922 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:00.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:00.931 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.637468095000003, 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:00.932 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:38:00.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:38:00.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:00.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:38:00.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.630711073999997, 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:00.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.625387414999999, 2024-09-21T23:38:00.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:38:00.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:38:00.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.952 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.624519804999998, 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:00.953 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:38:00.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:38:00.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:38:00.957 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.640321747999998, 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:38:00.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:38:00.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:38:00.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:38:00.964 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.637620519000002, 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:38:00.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.625884608, 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:00.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:38:00.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:38:00.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.619131977999999, 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:00.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:00.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:00.981 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:00.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:38:00.984 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 0.870786 (8570672) 2024-09-21T23:38:00.984 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:00.982+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:00.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:01.084 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:01.082+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:01.108 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:01.108 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:01.108 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:01.108 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:01.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:01.210 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:01.210+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:01.272 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:01.273 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:01.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:01.286 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:01.286+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:01.387 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:01.386+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:01.443 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.472531556, 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.437376118000003, 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.423905388000001, 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.400231290999997, 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.336904330000003, 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.164734336000002, 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.814095041000002, 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.813387253000002, 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.812839744000001, 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.812187023, 2024-09-21T23:38:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.811231413000002, 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.810574715000001, 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.810042959, 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.809229299000002, 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.808757395999997, 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.807727763999999, 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.807105114999999, 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.801519376000002, 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.800559106000001, 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.799798271999997, 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:01.535 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:01.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:01.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:01.537 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:01.534+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:01.637 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:01.634+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:01.699 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:01.700 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:01.700 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:01.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:01.713 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:01.713 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:01.728 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:01.728 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:01.738 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:01.742+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:01.744 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:01.744 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:01.764 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:01.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:01.784 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:01.784 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:01.804 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:01.839 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:01.839 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:01.839 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:01.842+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:01.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:01.940 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:01.942+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:01.963 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:38:02.018 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.018 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:02.042 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:02.042+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:02.142 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:02.142+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:02.189 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.189 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:02.246 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:02.246+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:02.344 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:02.342+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:02.362 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.363 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:02.462 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:02.462+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:02.538 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.538 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.545 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:02.542+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:02.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:02.647 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:02.646+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:02.681 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.681 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.695 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:02.747 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:02.750+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:02.813 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.814 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:02.829 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:02.848 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:02.846+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:02.949 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:02.946+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:02.998 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:02.998 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:03.058 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:03.062+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:03.129 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.129 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.143 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:03.150 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:03.150+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:03.251 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:03.250+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:03.308 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.308 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.323 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:03.366 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:03.370+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:03.439 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.439 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.455 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:03.455 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:03.458+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:03.555 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:03.554+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:03.621 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.623 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:03.657 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:03.654+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:03.755 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.755 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.757 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:03.758+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:03.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:03.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:03.866 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:03.866+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:03.935 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:03.935 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:03.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:03.973 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:03.973 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:03.974+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:03.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:38:03.990 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:03.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:04.008 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:04.008 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:04.028 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:04.028 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:04.049 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:04.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:04.061 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:04.062+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:04.067 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:04.067 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.069 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:04.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:04.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:04.177 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:04.178+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:04.237 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.238 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:04.238 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:04.238 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:04.238 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:04.262 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:04.266+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:04.363 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:04.366+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:04.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.4370057, 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:04.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:04.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.401850262000004, 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:04.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.388379532000002, 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:04.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:04.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.364705434999998, 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:04.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:04.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.301378474000003, 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.129208480000003, 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.778569185000002, 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:04.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:04.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:04.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:04.442 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.777861397000002, 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:04.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.777313888000002, 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:04.447 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:04.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:04.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.776661167, 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:04.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:04.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:04.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.775705557000002, 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:04.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.775048859000002, 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:04.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:04.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.774517103000001, 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:04.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:04.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:04.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.773703443000002, 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.773231539999998, 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.772201908, 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.771579258999999, 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.765993520000002, 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.765033250000002, 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.764272415999997, 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.500 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:04.502 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:04.506+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:04.602 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:04.606+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:04.629 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.629 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:04.629 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:04.629 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:04.703 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:04.702+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:04.759 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:04.759 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:04.804 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:04.806+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:04.905 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:04.902+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:04.937 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.678007190999999, 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:38:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:38:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:38:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.659919264999999, 2024-09-21T23:38:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:38:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:38:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:04.946 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:04.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:04.976 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:38:05.006 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:05.006+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:05.107 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:05.106+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:05.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:05.208 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:05.210+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:05.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:05.278 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:05.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:05.338 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:05.338+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:05.410 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:05.410+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:05.416 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.150025333999999, 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:38:05.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:38:05.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:38:05.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:05.421 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:05.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:05.512 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:05.514+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:05.562 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:05.562 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:05.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:05.619 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:05.622+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:05.714 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:05.714+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:05.743 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:05.743 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:05.743 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:05.743 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:05.743 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:05.757 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:05.814 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:05.814+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.649824664, 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.880 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:05.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.880639209000002, 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:05.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:05.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:05.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:05.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.880507686000001, 2024-09-21T23:38:05.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:05.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:05.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.880447777000001, 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:05.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.903 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.779620500999997, 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:05.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:05.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.779505012000001, 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:05.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:05.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:05.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.916 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.613233299000001, 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.917 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:05.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:05.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:05.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.613186388999999, 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:05.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:05.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.613145003, 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:05.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:05.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.612968131000002, 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.934 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:05.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:05.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.612920434999999, 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.612560250999998, 2024-09-21T23:38:05.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:05.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:05.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:05.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:05.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:05.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.612432294999998, 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:05.953 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:05.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:05.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.612395018999997, 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:05.959 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.248635614000001, 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.965 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.970 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.248600854000003, 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:05.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.976 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.248504928000003, 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.633783477999998, 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:05.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.633724622000003, 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.988 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:05.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:05.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:05.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.633611428999998, 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.994 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:05.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:05.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:05.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:05.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:05.999 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:06.000 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:06.000 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:06.000 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:38:06.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:38:06.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:06.002 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:05.998+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:06.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:06.102 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:06.106+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:06.130 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:06.130 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:06.145 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:06.203 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:06.202+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:06.246 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:06.246 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:06.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:06.263 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:06.279 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:06.279 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:06.299 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:06.299 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:06.304 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:06.306+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:06.309 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 42, 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66973394-6d88-4f14-bf11-7ccd7c9cf4a7", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-21T23:36:58.086363+0000", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-21T23:38:04.970334+0000", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-21T23:37:09.181667+0000", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-21T23:36:59.825771+0000", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-21T23:38:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 6, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "squid", 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-21T23:38:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:11.018666+0000", 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:38:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:38:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "16", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:38:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:38:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.97846227884292603, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:15.320283+0000", 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:38:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:38:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 2, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 20, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:38:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:38:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-21T23:38:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.25, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.25, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.25, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.25, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.97846227884292603, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:38:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "638e3e48-1763-4ec9-8c06-634e0b3b793b", 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/915397992", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/915397992", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/915397992", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/915397992", 2024-09-21T23:38:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "02967526-d5c5-4f68-8e25-343311dbaec4", 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:38:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/3900686792", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/3900686792", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/3900686792", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/3900686792", 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "4f770ac1-d062-4934-81d5-7f980b3830a1", 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.8707733154296875, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:38:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/2564679849", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/2564679849", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6823/2564679849", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/2564679849", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "90cc3313-3845-48d0-839d-8fea4c52bfde", 2024-09-21T23:38:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6808", 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6809", 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6810", 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6811", 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6814", 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6815", 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6812", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6813", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6809/1410364236", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6811/1410364236", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6815/1410364236", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6813/1410364236", 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "42b8e641-af88-4d41-bc5f-a14020f205ce", 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2024-09-21T23:38:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6800", 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6801", 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6802", 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6803", 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6806", 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6807", 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6804", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6805", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6801/1572346429", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6803/1572346429", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6807/1572346429", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6805/1572346429", 2024-09-21T23:38:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9", 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:38:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6816", 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6817", 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6818", 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6819", 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6822", 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6823", 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:38:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6820", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6821", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6817/466572374", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6819/466572374", 2024-09-21T23:38:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6823/466572374", 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6821/466572374", 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.810277+0000", 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.686068+0000", 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:38:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.949768+0000", 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.773303+0000", 2024-09-21T23:38:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.009610+0000", 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:38:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.059224+0000", 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:06.361 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:06.362 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:06.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:06.363 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:06.363 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:06.363 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:06.363 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:06.363 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:06.364 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:06.365 INFO:tasks.thrashosds.thrasher:j is {'epoch': 42, 'fsid': '66973394-6d88-4f14-bf11-7ccd7c9cf4a7', 'created': '2024-09-21T23:36:58.086363+0000', 'modified': '2024-09-21T23:38:04.970334+0000', 'last_up_change': '2024-09-21T23:37:09.181667+0000', 'last_in_change': '2024-09-21T23:36:59.825771+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:37:11.018666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.978462278842926, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:37:15.320283+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.978462278842926, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '638e3e48-1763-4ec9-8c06-634e0b3b793b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 915397992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 915397992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 915397992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 915397992}]}, 'public_addr': '172.21.15.79:6801/915397992', 'cluster_addr': '172.21.15.79:6803/915397992', 'heartbeat_back_addr': '172.21.15.79:6807/915397992', 'heartbeat_front_addr': '172.21.15.79:6805/915397992', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02967526-d5c5-4f68-8e25-343311dbaec4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 3900686792}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 3900686792}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 3900686792}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 3900686792}]}, 'public_addr': '172.21.15.79:6809/3900686792', 'cluster_addr': '172.21.15.79:6811/3900686792', 'heartbeat_back_addr': '172.21.15.79:6815/3900686792', 'heartbeat_front_addr': '172.21.15.79:6813/3900686792', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4f770ac1-d062-4934-81d5-7f980b3830a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8707733154296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2564679849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 2564679849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 2564679849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 2564679849}]}, 'public_addr': '172.21.15.79:6817/2564679849', 'cluster_addr': '172.21.15.79:6819/2564679849', 'heartbeat_back_addr': '172.21.15.79:6823/2564679849', 'heartbeat_front_addr': '172.21.15.79:6821/2564679849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '90cc3313-3845-48d0-839d-8fea4c52bfde', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 1410364236}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 1410364236}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 1410364236}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 1410364236}]}, 'public_addr': '172.21.15.118:6809/1410364236', 'cluster_addr': '172.21.15.118:6811/1410364236', 'heartbeat_back_addr': '172.21.15.118:6815/1410364236', 'heartbeat_front_addr': '172.21.15.118:6813/1410364236', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '42b8e641-af88-4d41-bc5f-a14020f205ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 1572346429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 1572346429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 1572346429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 1572346429}]}, 'public_addr': '172.21.15.118:6801/1572346429', 'cluster_addr': '172.21.15.118:6803/1572346429', 'heartbeat_back_addr': '172.21.15.118:6807/1572346429', 'heartbeat_front_addr': '172.21.15.118:6805/1572346429', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6817', 'nonce': 466572374}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6818', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 466572374}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6822', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 466572374}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6821', 'nonce': 466572374}]}, 'public_addr': '172.21.15.118:6817/466572374', 'cluster_addr': '172.21.15.118:6819/466572374', 'heartbeat_back_addr': '172.21.15.118:6823/466572374', 'heartbeat_front_addr': '172.21.15.118:6821/466572374', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.810277+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.686068+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.949768+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.773303+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.009610+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.059224+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.0', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:38:06.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:38:06.365 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:06.366 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:06.406 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:06.406+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:06.506 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:06.506+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:06.535 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:06.535 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:06.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:06.606 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:06.606+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:06.686 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:06.686 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:38:06.702 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T23:38:04.977777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":194904,"kb_used_data":3604,"kb_used_omap":9,"kb_used_meta":191158,"kb_avail":566036136,"statfs":{"total":579820584960,"available":579621003264,"internally_reserved":0,"allocated":3690496,"data_stored":2069779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":195746465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.016244"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.930048+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:37:55.930048+0000","last_peered":"2024-09-21T23:37:55.930048+0000","last_clean":"2024-09-21T23:37:55.930048+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:37:55.930048+0000","last_undegraded":"2024-09-21T23:37:55.930048+0000","last_fullsized":"2024-09-21T23:37:55.930048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:09.310359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T23:37:58.942451+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:37:58.942451+0000","last_peered":"2024-09-21T23:37:58.942451+0000","last_clean":"2024-09-21T23:37:58.942451+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:37:58.942451+0000","last_undegraded":"2024-09-21T23:37:58.942451+0000","last_fullsized":"2024-09-21T23:37:58.942451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:13.117249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.930009+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:37:55.930009+0000","last_peered":"2024-09-21T23:37:55.930009+0000","last_clean":"2024-09-21T23:37:55.930009+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:37:55.930009+0000","last_undegraded":"2024-09-21T23:37:55.930009+0000","last_fullsized":"2024-09-21T23:37:55.930009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:17.943327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T23:37:58.942458+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:37:58.942458+0000","last_peered":"2024-09-21T23:37:58.942458+0000","last_clean":"2024-09-21T23:37:58.942458+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:37:58.942458+0000","last_undegraded":"2024-09-21T23:37:58.942458+0000","last_fullsized":"2024-09-21T23:37:58.942458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:51:17.717371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":57,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.933317+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:37:55.933317+0000","last_peered":"2024-09-21T23:37:55.933317+0000","last_clean":"2024-09-21T23:37:55.933317+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:37:55.933317+0000","last_undegraded":"2024-09-21T23:37:55.933317+0000","last_fullsized":"2024-09-21T23:37:55.933317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:11:25.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.949642+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:37:55.949642+0000","last_peered":"2024-09-21T23:37:55.949642+0000","last_clean":"2024-09-21T23:37:55.949642+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:37:55.949642+0000","last_undegraded":"2024-09-21T23:37:55.949642+0000","last_fullsized":"2024-09-21T23:37:55.949642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:43.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.958967+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:37:55.958967+0000","last_peered":"2024-09-21T23:37:55.958967+0000","last_clean":"2024-09-21T23:37:55.958967+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:37:55.958967+0000","last_undegraded":"2024-09-21T23:37:55.958967+0000","last_fullsized":"2024-09-21T23:37:55.958967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:03.664659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":56,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.933356+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:37:55.933356+0000","last_peered":"2024-09-21T23:37:55.933356+0000","last_clean":"2024-09-21T23:37:55.933356+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:37:55.933356+0000","last_undegraded":"2024-09-21T23:37:55.933356+0000","last_fullsized":"2024-09-21T23:37:55.933356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.388030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T23:37:55.930008+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:37:55.930008+0000","last_peered":"2024-09-21T23:37:55.930008+0000","last_clean":"2024-09-21T23:37:55.930008+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:37:55.930008+0000","last_undegraded":"2024-09-21T23:37:55.930008+0000","last_fullsized":"2024-09-21T23:37:55.930008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607566,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35756,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336084,"statfs":{"total":96636764160,"available":96600150016,"internally_reserved":0,"allocated":413696,"data_stored":145887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607567,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35760,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":421888,"data_stored":150467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607566,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1003520,"data_stored":736236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607566,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":409600,"data_stored":145868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607565,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":425984,"data_stored":150486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607566,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1015808,"data_stored":740835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:38:06.703 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [5, 1, 4, 3] 2024-09-21T23:38:06.703 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.0', '5', '1', '4', '3'] 2024-09-21T23:38:06.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.0 5 1 4 3 2024-09-21T23:38:06.716 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:06.716 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:06.731 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:06.746 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:06.746+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:06.808 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:06.806+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:06.853 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:06.853 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:06.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:06.910 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:06.914+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:06.984 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:06.985 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:06.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:38:06.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:07.043 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:07.046+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:07.108 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.108 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.110 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:07.114+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:07.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:07.211 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:07.210+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:07.228 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.228 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:07.312 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:07.314+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:07.408 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.408 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.414 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:07.414+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:07.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:07.514 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:07.514+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:07.538 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.539 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:07.615 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:07.614+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:07.670 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.670 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:07.715 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:07.718+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:07.816 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:07.818+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:07.850 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.850 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:07.917 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:07.918+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:07.981 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:07.981 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:07.994 INFO:teuthology.orchestra.run.smithi079.stderr:set 1.0 pg_upmap_items mapping to [5->1,4->3] 2024-09-21T23:38:07.994 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:38:07.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:08.018 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:08.018+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:08.111 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:08.111 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:08.119 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:08.118+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:08.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:08.220 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:08.222+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:08.289 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:08.290 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:08.304 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:08.321 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:08.322+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:08.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:08.422 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:08.422+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:08.434 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:08.434 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:08.449 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:08.516 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:08.516 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:08.523 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:08.522+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:08.532 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:08.532 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:08.548 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:08.549 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:08.569 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:08.570 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:08.572 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:08.572 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:08.572 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:08.572 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:08.572 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:08.586 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:08.587 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:08.587 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:08.605 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:08.605 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:08.635 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:08.638+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.482279636000001, 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.713094181000002, 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.712962658000002, 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.712902749000001, 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.612075472999997, 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.611959984000002, 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445688271000002, 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445641361, 2024-09-21T23:38:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445599975, 2024-09-21T23:38:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445423103000003, 2024-09-21T23:38:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445375407, 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.445015222999999, 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.444887266999999, 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.444849990999998, 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.081090586000002, 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.081055825999997, 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.080959900000003, 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.466238449999999, 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.466179594000003, 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:08.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.466066400999999, 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:08.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:08.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:08.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:08.831 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:08.833 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:08.833 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:08.834+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:08.934 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:08.934+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:09.004 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.004 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:09.004 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:09.004 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:09.034 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:09.034+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:09.135 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:09.138+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:09.140 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:09.141 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.155 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:09.242 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:09.246+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:09.319 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.319 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:09.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:09.319 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:38:09.319 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.333 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:09.375 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:09.378+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:09.437 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:09.438+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:09.497 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.498 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:09.498 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:09.498 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.512 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:09.555 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:09.558+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:09.639 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:09.642+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:09.678 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.679 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:09.679 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:09.679 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:09.679 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.693 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:09.740 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:09.742+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:09.841 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:09.842+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:09.862 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.599335955000001, 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:09.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:38:09.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:09.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:38:09.869 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.592578934000002, 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.587255274999997, 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.586387664999997, 2024-09-21T23:38:09.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:38:09.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.602189608000003, 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:38:09.881 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.599488379, 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:09.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:09.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:38:09.890 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.587752467999998, 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:09.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:38:09.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:38:09.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:38:09.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:09.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.580999837999997, 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:09.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:38:09.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:38:09.900 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:09.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:09.901 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:09.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:09.943 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:09.946+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:10.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:38:10.035 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:10.035 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:10.035 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:10.035 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:10.042 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:10.042+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:10.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:10.142 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:10.142+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:10.215 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:10.215 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:10.215 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:10.215 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:10.215 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:10.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:10.243 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:10.246+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:10.344 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:10.342+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:10.391 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.419648678000001, 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.384493239999998, 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.371022510000003, 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.347348412999999, 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.284021451999998, 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.111851457999997, 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.761212163000003, 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.760504374999996, 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.759956866000003, 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.759304145000002, 2024-09-21T23:38:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.758348535000003, 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.757691837000003, 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.757160081000002, 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.756346421000003, 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.755874517999999, 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.754844886000001, 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.754222237, 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.748636498000003, 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.747676228000003, 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.746915393999998, 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:10.478 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:10.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:10.480 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:10.482+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:10.581 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:10.578+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:10.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:10.649 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:10.650 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:10.650 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:10.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:10.682 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:10.682+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:10.768 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:10.769 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:10.782 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:10.782+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:10.785 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:10.803 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:10.803 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:10.823 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:10.824 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:10.827 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:10.827 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:10.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:10.844 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:10.844 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:10.865 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:10.865 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:10.882 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:10.886+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:10.956 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:10.957 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:10.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:11.010 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:38:11.010 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:11.010+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:11.084 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:11.082+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:11.090 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.090 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:11.186 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:11.190+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:11.261 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.262 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:11.321 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:11.322+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:11.388 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:11.390+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:11.397 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.397 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.411 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:11.488 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:11.490+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:11.541 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.541 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:11.595 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:11.598+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:11.690 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:11.690+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:11.719 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.719 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.733 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:11.790 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:11.790+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:11.855 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.855 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.869 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:11.919 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:11.922+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:11.992 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:11.992 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:11.992 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:11.990+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:12.006 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:12.095 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:12.098+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:12.184 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.184 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.193 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:12.194+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:12.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:12.294 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:12.294+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:12.367 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.367 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:12.395 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:12.394+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:12.496 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:12.494+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:12.504 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.504 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.519 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:12.597 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:12.598+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.652 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:12.739 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:12.742+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:12.798 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:12.798+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:12.820 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.820 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:12.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:12.899 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:12.902+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:12.967 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:12.968 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:12.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:13.000 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:13.002+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:13.022 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:13.022 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:13.023 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:38:13.023 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:38:13.023 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:38:13.023 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:38:13.033 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:38:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:13.042 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:13.059 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:13.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:13.077 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:13.078 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:13.097 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:13.097 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:13.101 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:13.098+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:13.112 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.113 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:13.113 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:13.113 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:13.113 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:13.116 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:13.116 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:13.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:13.136 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:13.136 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:13.154 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:38:13.154 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:13.201 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:13.202+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:13.255 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:13.255 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.283949598, 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.248794160000003, 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:13.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:13.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.235323430000001, 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:13.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.211649332999997, 2024-09-21T23:38:13.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:13.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:13.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.148322372000003, 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:13.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:13.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.976152378000002, 2024-09-21T23:38:13.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.625513083000001, 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:13.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:13.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.624805295000002, 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:13.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.624257786000001, 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:13.292 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.623605065, 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:13.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:13.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.622649455000001, 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:13.301 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:13.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.621992757000001, 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.306 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:13.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:13.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.621461001, 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:13.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:13.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.620647341000002, 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.315 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:13.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.620175437999997, 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:13.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.619145805999999, 2024-09-21T23:38:13.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:13.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:13.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.618523156999998, 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.612937418000001, 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.611977148000001, 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:13.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.611216314000004, 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:13.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.347 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:13.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:13.349 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:13.350+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:13.449 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:13.446+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:13.508 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.508 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:13.508 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:13.508 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:13.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:13.550 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:13.550+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:13.651 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:13.654+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:13.678 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:13.679 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:13.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:13.752 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:13.750+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:13.809 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.549963968999997, 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:38:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:38:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.531876042999997, 2024-09-21T23:38:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:38:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:38:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:13.818 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:13.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:13.865 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:13.866+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:13.953 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:13.954+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:13.988 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:13.988 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:14.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:14.014 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:38:14.054 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:14.054+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:14.135 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:14.135 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:14.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:14.155 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:14.158+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:14.255 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:14.254+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.001695032999997, 2024-09-21T23:38:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:14.273 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:14.281 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:14.356 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:14.354+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:14.415 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:14.415 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:14.415 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:14.415 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:14.429 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:14.458 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:14.458+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:14.547 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:14.547 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:14.559 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:14.562+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:14.562 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:14.660 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:14.658+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:14.731 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:14.731 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:14.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:14.731 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.501299338999999, 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.732113884, 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:14.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:14.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:14.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:14.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.731982361, 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:14.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:14.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.731922451999999, 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:14.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:14.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.631095176000002, 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.630979687, 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464707974, 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464661063999998, 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464619677999998, 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464442806000001, 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464395109999998, 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.464034925999997, 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.463906969999996, 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.463869694000003, 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.100110289, 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.100075529000001, 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.099979603000001, 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.485258152999997, 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.485199297000001, 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.485086103999997, 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:14.850 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:14.852 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:14.852 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:14.854+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:14.952 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:14.954+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:15.022 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:15.022 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:15.022 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:15.022 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:15.037 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:15.052 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:15.050+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:15.153 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:15.154+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:15.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:15.204 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:15.204 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:15.204 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:15.204 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:15.204 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:15.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:15.263 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:15.265+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:15.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:15.309 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:15.326 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:15.326 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:15.341 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:15.341 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:15.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:15.341 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:38:15.341 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:15.342 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:15.342 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:15.354 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:15.353+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:15.355 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:15.363 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:15.363 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:15.383 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:15.401 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:15.455 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:15.457+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:15.522 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:15.522 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:15.537 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:15.556 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:15.554+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:15.657 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:15.658+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:15.701 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:15.701 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:15.715 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:15.758 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:15.758+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:15.833 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:15.833 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:15.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:15.858 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:15.857+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:15.958 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:15.957+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:15.977 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:15.978 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:15.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:16.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:38:16.059 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:16.061+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:16.117 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.117 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:16.160 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:16.161+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:16.263 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:16.262+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:16.268 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.268 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:16.370 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:16.370+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:16.447 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.447 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:16.463 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:16.465+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:16.564 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:16.565+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:16.629 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.629 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:16.665 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:16.665+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:16.766 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:16.766+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:16.768 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.768 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:16.867 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:16.869+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:16.946 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:16.946 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:16.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:16.968 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:16.969+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:17.032 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:38:17.069 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:17.069+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:17.087 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:17.087 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:17.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:17.170 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:17.169+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:17.271 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:17.271 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:17.271 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:17.273+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:17.285 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:17.372 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:17.370+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:17.417 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:17.417 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:17.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:17.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:17.475 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:17.477+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:17.549 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:17.549 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:17.564 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:17.568 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:17.586 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:17.586 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:17.601 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:17.602 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:17.607 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:17.609+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:17.620 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:17.639 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:17.639 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:17.659 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:17.659 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:17.673 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:17.674+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.505658065000006, 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.736472610000007, 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.736341087, 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.736281177999999, 2024-09-21T23:38:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.635453901999995, 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.635338412999999, 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.469066699999999, 2024-09-21T23:38:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.469019790000004, 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468978403999998, 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468801532000001, 2024-09-21T23:38:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468753836000005, 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468393652000003, 2024-09-21T23:38:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468265696000003, 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.468228420000003, 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.104469015000006, 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.104434255000001, 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.104338329000001, 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.489616879000003, 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.489558023000001, 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.489444829999997, 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:17.858 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:17.860 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:17.860 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:17.861+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:17.960 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:17.958+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:18.030 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.030 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:18.030 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:18.030 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.045 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:18.091 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:18.093+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:18.162 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:18.161+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.163 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:18.163 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:18.163 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.169 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:18.169 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:18.169 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.571301321361582 2024-09-21T23:38:18.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.571301321361582 2024-09-21T23:38:18.177 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:18.264 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:18.265+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:18.345 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [] 2024-09-21T23:38:18.345 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.359 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:18.364 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:18.362+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:18.465 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:18.465+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:Note: switching to '4e45e6b490e6cbe082cdf155fea780438ec53fc7'. 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr: git switch -c 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.509 INFO:tasks.workunit.client.0.smithi165.stderr:Or undo this operation with: 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr: git switch - 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr: 2024-09-21T23:38:18.510 INFO:tasks.workunit.client.0.smithi165.stderr:HEAD is now at 4e45e6b490e Merge pull request #59907 from zdover23/wip-doc-2024-09-21-backport-59904-to-squid 2024-09-21T23:38:18.516 DEBUG:teuthology.orchestra.run.smithi165:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T23:38:18.528 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.528 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:18.528 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:18.528 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.542 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:18.566 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:18.566+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:18.566 INFO:tasks.workunit.client.0.smithi165.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T23:38:18.569 INFO:tasks.workunit.client.0.smithi165.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T23:38:18.569 INFO:tasks.workunit.client.0.smithi165.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T23:38:18.667 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:18.669+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:18.705 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.705 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:18.705 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:18.706 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:18.706 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.719 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:18.807 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:18.809+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:18.869 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:18.866+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.622701153000001, 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0073273230000000002, 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266686+0000", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266693+0000", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1230000000000002e-06 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266703+0000", 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0247e-05 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266730+0000", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7290000000000001e-05 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.266757+0000", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6203e-05 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267562+0000", 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080495000000000002 2024-09-21T23:38:18.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.267584+0000", 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1917e-05 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271073+0000", 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034893960000000001 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271321+0000", 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024821799999999997 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.271608+0000", 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028670799999999999 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273681+0000", 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020729799999999999 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273764+0000", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3526e-05 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273897+0000", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013218299999999999 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.274013+0000", 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116582 2024-09-21T23:38:18.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.615944132000003, 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025286500000000001, 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273443+0000", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273451+0000", 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2199999999999992e-06 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273462+0000", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0609e-05 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273537+0000", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5104000000000005e-05 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273563+0000", 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5806e-05 2024-09-21T23:38:18.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273660+0000", 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7572999999999996e-05 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273672+0000", 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1996e-05 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.273696+0000", 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3557000000000002e-05 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.610620472999997, 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043734299999999998, 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:18.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 2 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278766+0000", 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278769+0000", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278779+0000", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0406e-05 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278804+0000", 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4902999999999999e-05 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278825+0000", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0950000000000001e-05 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.278988+0000", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000163165 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279204+0000", 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021579499999999999 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.900 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.609752862999997, 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041597699999999999, 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3 2024-09-21T23:38:18.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279634+0000", 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279636+0000", 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8500000000000001e-06 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279644+0000", 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279667+0000", 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-05 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279693+0000", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6060000000000001e-05 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.279872+0000", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000179135 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:17.280050+0000", 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178209 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.904 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.625554805999997, 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030764630000000002, 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 5 2024-09-21T23:38:18.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263832+0000", 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263835+0000", 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263850+0000", 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263879+0000", 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8979000000000002e-05 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.263904+0000", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5187000000000001e-05 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264064+0000", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160056 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264364+0000", 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030016999999999999 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.264576+0000", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021158900000000001 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266728+0000", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021524000000000001 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266766+0000", 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7786e-05 2024-09-21T23:38:18.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266843+0000", 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7043000000000001e-05 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266909+0000", 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5772999999999995e-05 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:5 2.2 e19/17 ondisk, result = 0)", 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.622853577000001, 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021151500000000001, 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266533+0000", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266535+0000", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8169999999999999e-06 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266544+0000", 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266570+0000", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5469e-05 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266588+0000", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8777e-05 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266706+0000", 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117699 2024-09-21T23:38:18.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266719+0000", 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.294e-05 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:18.266745+0000", 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5763e-05 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4212.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.611117665999998, 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071851409999999999, 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4212", 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.79:0/2351611241", 2024-09-21T23:38:18.915 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 7 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278269+0000", 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278273+0000", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.963e-06 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278292+0000", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8783e-05 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278337+0000", 2024-09-21T23:38:18.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5460000000000002e-05 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278364+0000", 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6313e-05 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278539+0000", 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017479799999999999 2024-09-21T23:38:18.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.278557+0000", 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8442999999999999e-05 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282301+0000", 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003744392 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282466+0000", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000164222 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.282698+0000", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023243100000000001 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.284172+0000", 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001474382 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285287+0000", 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001114681 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285380+0000", 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2706000000000003e-05 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285454+0000", 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4567000000000006e-05 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4212.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.604365035999997, 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044798400000000001, 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:18.922 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285022+0000", 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285024+0000", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9570000000000001e-06 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285033+0000", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285062+0000", 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285086+0000", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.372e-05 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285261+0000", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017499799999999999 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285278+0000", 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:18.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:19.285470+0000", 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019183899999999999 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:18.926 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:18.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:18.977 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:18.978+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:19.037 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 0.571301 (9240) 2024-09-21T23:38:19.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:38:19.070 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.070 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:19.070 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:19.070 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.070 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:19.070+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:19.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:19.172 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:19.173+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:19.233 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:19.234 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:19.272 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:19.273+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:19.363 INFO:tasks.workunit.client.0.smithi165.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T23:38:19.364 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.394115959000004, 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.358960521, 2024-09-21T23:38:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.345489791000006, 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.321815693999994, 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.258488732999993, 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.086318739000006, 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.735679443999999, 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.734971655999999, 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.734424146999999, 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.733771426000004, 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.732815815999999, 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.732159117999998, 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.731627361999998, 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.730813702000006, 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.730341799000001, 2024-09-21T23:38:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.729312167000003, 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.728689517999996, 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.723103778999999, 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.722143509000006, 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 65.721382675000001, 2024-09-21T23:38:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.455 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.456 INFO:tasks.workunit.client.0.smithi165.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T23:38:19.456 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:19.457 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:19.457+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:19.489 INFO:tasks.workunit.client.0.smithi165.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T23:38:19.491 INFO:tasks.workunit.client.0.smithi165.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T23:38:19.491 INFO:tasks.workunit.client.0.smithi165.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T23:38:19.539 INFO:tasks.workunit.client.0.smithi165.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T23:38:19.542 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T23:38:19.542 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T23:38:19.557 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:19.554+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:19.583 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.583 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:19.583 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:19.583 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.589 INFO:tasks.workunit:Running workunits matching rbd/rbd-nbd.sh on client.0... 2024-09-21T23:38:19.590 INFO:tasks.workunit:Running workunit rbd/rbd-nbd.sh... 2024-09-21T23:38:19.591 DEBUG:teuthology.orchestra.run.smithi165:workunit test rbd/rbd-nbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4e45e6b490e6cbe082cdf155fea780438ec53fc7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh 2024-09-21T23:38:19.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:19.642 INFO:tasks.workunit.client.0.smithi165.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh 2024-09-21T23:38:19.643 INFO:tasks.workunit.client.0.smithi165.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2024-09-21T23:38:19.643 INFO:tasks.workunit.client.0.smithi165.stderr:++ TIMEOUT=300 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ MAX_TIMEOUT=15 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ PG_NUM=4 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ TMPDIR=/tmp 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ TESTDIR=/home/ubuntu/cephtest 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ type xmlstarlet 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:++ XMLSTARLET=xmlstarlet 2024-09-21T23:38:19.644 INFO:tasks.workunit.client.0.smithi165.stderr:+++ uname 2024-09-21T23:38:19.645 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' Linux = FreeBSD ']' 2024-09-21T23:38:19.645 INFO:tasks.workunit.client.0.smithi165.stderr:++ SED=sed 2024-09-21T23:38:19.645 INFO:tasks.workunit.client.0.smithi165.stderr:++ AWK=awk 2024-09-21T23:38:19.646 INFO:tasks.workunit.client.0.smithi165.stderr:+++ stty -a 2024-09-21T23:38:19.646 INFO:tasks.workunit.client.0.smithi165.stderr:+++ head -1 2024-09-21T23:38:19.646 INFO:tasks.workunit.client.0.smithi165.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2024-09-21T23:38:19.655 INFO:tasks.workunit.client.0.smithi165.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-09-21T23:38:19.655 INFO:tasks.workunit.client.0.smithi165.stderr:++ termwidth= 2024-09-21T23:38:19.655 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2024-09-21T23:38:19.656 INFO:tasks.workunit.client.0.smithi165.stderr:++ DIFFCOLOPTS='-y ' 2024-09-21T23:38:19.656 INFO:tasks.workunit.client.0.smithi165.stderr:++ KERNCORE=kernel.core_pattern 2024-09-21T23:38:19.656 INFO:tasks.workunit.client.0.smithi165.stderr:++ EXTRA_OPTS= 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:++ test '' = TESTS 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ POOL=rbd 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ ANOTHER_POOL=new_default_pool21555 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ NS=ns 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ IMAGE=testrbdnbd21555 2024-09-21T23:38:19.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ SIZE=64 2024-09-21T23:38:19.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ DATA= 2024-09-21T23:38:19.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:19.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ setup 2024-09-21T23:38:19.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns x 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -e CMakeCache.txt ']' 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo echo test sudo 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:19.662 INFO:tasks.workunit.client.0.smithi165.stderr:++ which echo 2024-09-21T23:38:19.664 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/echo 2024-09-21T23:38:19.664 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:19.664 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/echo test sudo 2024-09-21T23:38:19.671 INFO:tasks.workunit.client.0.smithi165.stdout:test sudo 2024-09-21T23:38:19.672 INFO:tasks.workunit.client.0.smithi165.stderr:+ trap cleanup INT TERM EXIT 2024-09-21T23:38:19.673 INFO:tasks.workunit.client.0.smithi165.stderr:++ mktemp -d 2024-09-21T23:38:19.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:19.681 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:19.682+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:19.699 INFO:tasks.workunit.client.0.smithi165.stderr:+ TEMPDIR=/tmp/tmp.h0CUjCtN4b 2024-09-21T23:38:19.699 INFO:tasks.workunit.client.0.smithi165.stderr:+ DATA=/tmp/tmp.h0CUjCtN4b/data 2024-09-21T23:38:19.699 INFO:tasks.workunit.client.0.smithi165.stderr:+ dd if=/dev/urandom of=/tmp/tmp.h0CUjCtN4b/data bs=1M count=64 2024-09-21T23:38:19.759 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:19.758+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:19.759 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:19.759 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:19.817 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:19.818 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:19.861 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:19.862+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:19.894 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:19.894 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:19.895 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.194444 s, 345 MB/s 2024-09-21T23:38:19.895 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd namespace create rbd/ns 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.670856297999997, 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:38:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:38:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.652768371999997, 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:38:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:38:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:19.939 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:19.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:19.985 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:19.986+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:19.988 INFO:tasks.workunit.client.0.smithi165.stderr:+ for ns in '' ${NS} 2024-09-21T23:38:19.988 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd --dest-pool rbd --dest-namespace '' --no-progress import /tmp/tmp.h0CUjCtN4b/data testrbdnbd21555 2024-09-21T23:38:20.038 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:20.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:20.056 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:20.056 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:20.057 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:38:20.060 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:20.061+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:20.094 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:20.132 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.132 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.133 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:20.133 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:20.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:20.161 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:20.161+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:20.166 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:20.166 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:20.242 INFO:tasks.workunit.client.0.smithi165.stderr:+ for ns in '' ${NS} 2024-09-21T23:38:20.242 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd --dest-pool rbd --dest-namespace ns --no-progress import /tmp/tmp.h0CUjCtN4b/data testrbdnbd21555 2024-09-21T23:38:20.261 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:20.262+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:20.285 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.285 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:20.362 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:20.361+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:20.463 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:20.462+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:20.482 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.483 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:20.501 INFO:tasks.workunit.client.0.smithi165.stderr:+ ceph osd pool create new_default_pool21555 8 2024-09-21T23:38:20.564 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:20.562+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:20.621 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.621 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:20.666 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:20.666+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:20.757 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.758 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.766 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:20.766+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:20.771 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:20.867 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:20.869+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:20.890 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:20.890 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:20.903 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:20.995 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:20.997+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:21.070 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:21.070+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:21.071 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:21.071 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:21.086 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:21.158 INFO:tasks.workunit.client.0.smithi165.stderr:pool 'new_default_pool21555' already exists 2024-09-21T23:38:21.170 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:21.170+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:21.174 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd pool init new_default_pool21555 2024-09-21T23:38:21.251 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:21.252 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:21.266 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:21.271 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:21.270+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:21.372 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:21.373+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:21.547 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:21.547 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:21.547 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:21.548 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:21.546+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:21.648 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:21.646+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:21.718 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:21.719 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:21.733 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:21.779 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:21.781+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:21.850 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:21.849+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:21.850 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:21.850 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:21.864 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:21.951 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:21.953+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:22.030 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:22.030 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:22.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:22.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:38:22.093 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:22.094+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:22.152 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:22.150+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:22.181 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:22.181 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:22.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:22.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:22.285 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:22.286+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:22.329 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:22.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:22.346 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:22.346 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:22.353 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:22.353+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:22.357 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:22.358 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:22.362 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:22.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:22.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:22.383 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:22.384 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:22.404 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:22.405 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:22.425 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:22.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:22.455 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:22.453+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:22.498 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.528218916, 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034345729999999998, 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969826+0000", 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969827+0000", 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.46e-06 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969840+0000", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2713e-05 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969866+0000", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.969883+0000", 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-05 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970018+0000", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013517600000000001 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.970074+0000", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6122999999999998e-05 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973182+0000", 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031080130000000002 2024-09-21T23:38:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:12.973260+0000", 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8242000000000001e-05 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'18)", 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.493063477999996, 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002901804, 2024-09-21T23:38:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004981+0000", 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.004982+0000", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005012+0000", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005030+0000", 2024-09-21T23:38:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8626000000000001e-05 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005045+0000", 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5085999999999999e-05 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005139+0000", 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3599999999999998e-05 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.005172+0000", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2889000000000003e-05 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007782+0000", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026101430000000001 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.007883+0000", 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100796 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.479592748000002, 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.455918651000005, 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003281698, 2024-09-21T23:38:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042126+0000", 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042128+0000", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042182+0000", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4228999999999999e-05 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042212+0000", 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9280000000000001e-05 2024-09-21T23:38:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042225+0000", 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.326e-05 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042336+0000", 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110646 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.042388+0000", 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349000000000003e-05 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045322+0000", 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029344850000000001 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.045408+0000", 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5202000000000002e-05 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.392591690000003, 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028719679999999999, 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105453+0000", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105455+0000", 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105521+0000", 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6075999999999996e-05 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105545+0000", 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4119999999999999e-05 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105558+0000", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3353000000000001e-05 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105779+0000", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000220879 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.105817+0000", 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7611999999999998e-05 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108214+0000", 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023969350000000002 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.108325+0000", 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111203 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:115 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-21T23:38:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.220421696000002, 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003225983, 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277623+0000", 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277624+0000", 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.71e-07 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277734+0000", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110231 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277745+0000", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277756+0000", 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0679999999999999e-05 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277809+0000", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3344999999999998e-05 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.277832+0000", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2826000000000001e-05 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280807+0000", 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002974651 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.280849+0000", 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2284999999999999e-05 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.869782400999995, 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.869074612999995, 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.868527103999995, 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:134 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'85)", 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.867874383, 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031547400000000001, 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630170+0000", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630172+0000", 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3939999999999999e-06 2024-09-21T23:38:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630267+0000", 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5789000000000001e-05 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630289+0000", 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1699999999999999e-05 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630306+0000", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6572999999999999e-05 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630929+0000", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000623188 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630964+0000", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5379999999999997e-05 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633274+0000", 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023101860000000001 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633325+0000", 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0529999999999999e-05 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.866918772999995, 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.866262074999995, 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.865730318999994, 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.864916659000002, 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.864444755999997, 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.863415123999999, 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.862792475000006, 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:22.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:143 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'101, mlcod=16'98)", 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.857206735999995, 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030498470000000001, 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640838+0000", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640840+0000", 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640905+0000", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5833000000000005e-05 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640933+0000", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.76e-05 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.640951+0000", 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7623000000000001e-05 2024-09-21T23:38:22.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641164+0000", 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021361199999999999 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641197+0000", 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509000000000003e-05 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643836+0000", 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026388129999999998 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.643888+0000", 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2203000000000001e-05 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:144 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'102, mlcod=16'98)", 2024-09-21T23:38:22.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.856246466000002, 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034937459999999998, 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641798+0000", 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641800+0000", 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641861+0000", 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519000000000005e-05 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641888+0000", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7117000000000001e-05 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.641907+0000", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8836999999999999e-05 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642001+0000", 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3764999999999994e-05 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642030+0000", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9213999999999998e-05 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645189+0000", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031593519999999998 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645292+0000", 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102509 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:145 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'103, mlcod=16'98)", 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.855485631999997, 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032155420000000001, 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:22.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642559+0000", 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642560+0000", 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.127e-06 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642683+0000", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123022 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642724+0000", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0886000000000001e-05 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642741+0000", 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6823000000000001e-05 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642925+0000", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000184656 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.642976+0000", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0599999999999997e-05 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645726+0000", 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027500850000000002 2024-09-21T23:38:22.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.645775+0000", 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8343000000000001e-05 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:22.589 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:22.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:22.591 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:22.590+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:22.694 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:22.693+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:22.760 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:22.760 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:22.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:22.825 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:22.826+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:22.892 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:22.893 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:22.899 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:22.898+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:22.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:23.000 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:23.001+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.811713034999997, 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001475189, 2024-09-21T23:38:23.072 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260577+0000", 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260578+0000", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7600000000000006e-07 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260591+0000", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853999999999999e-05 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260645+0000", 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135999999999998e-05 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260662+0000", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7516e-05 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260748+0000", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5975000000000003e-05 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.260787+0000", 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8631999999999999e-05 2024-09-21T23:38:23.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.261993+0000", 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00120638 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:18.262052+0000", 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.872e-05 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 63.793625108999997, 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018229159999999999, 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278665+0000", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278666+0000", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278684+0000", 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7997000000000001e-05 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278712+0000", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7936000000000002e-05 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278727+0000", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5512999999999999e-05 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278875+0000", 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014834599999999999 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.278928+0000", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3075999999999998e-05 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280363+0000", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014342020000000001 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:19.280488+0000", 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125186 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4646.0:1 2.4 2:377edd40:::rbd_namespace:head [call rbd.namespace_add in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:19.972185+0000", 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.1001053249999999, 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024928929999999999, 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4646", 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2702222475", 2024-09-21T23:38:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972185+0000", 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972185+0000", 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972187+0000", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972214+0000", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7083000000000001e-05 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972237+0000", 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2656000000000001e-05 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972255+0000", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8317000000000001e-05 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972338+0000", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3007e-05 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972496+0000", 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015801699999999999 2024-09-21T23:38:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.972693+0000", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000196868 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.973778+0000", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010854879999999999 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974570+0000", 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079199800000000003 2024-09-21T23:38:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974629+0000", 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8400000000000003e-05 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974678+0000", 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8952000000000002e-05 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4646.0:1 2.4 e47/17 ondisk, result = 0)", 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:19.974450+0000", 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.0978398180000002, 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023648000000000001, 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974450+0000", 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974450+0000", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974452+0000", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5659999999999999e-06 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974460+0000", 2024-09-21T23:38:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4039999999999995e-06 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974484+0000", 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4213e-05 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974499+0000", 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4426999999999999e-05 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974558+0000", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9036999999999997e-05 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974565+0000", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.974687+0000", 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121796 2024-09-21T23:38:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4646.0:3 2.6 2:774f7988:ns::rbd_directory:head [call rbd.dir_state_set in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:19.982486+0000", 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.0898041219999999, 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030416140000000002, 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4646", 2024-09-21T23:38:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2702222475", 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982486+0000", 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982486+0000", 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982488+0000", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982498+0000", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0263e-05 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982527+0000", 2024-09-21T23:38:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8926000000000001e-05 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982541+0000", 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4007e-05 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982669+0000", 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128116 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.982954+0000", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028514799999999999 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.983488+0000", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053311199999999997 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.984279+0000", 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079191799999999996 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985474+0000", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011947710000000001 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985504+0000", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9416000000000001e-05 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985528+0000", 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3847000000000001e-05 2024-09-21T23:38:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4646.0:3 2.6 e47/17 ondisk, result = 0)", 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:19.985377+0000", 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.0869127509999998, 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015493500000000001, 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985377+0000", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985377+0000", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985378+0000", 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2999999999999999e-07 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985384+0000", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4450000000000004e-06 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985400+0000", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6756999999999999e-05 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985412+0000", 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1960999999999999e-05 2024-09-21T23:38:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985465+0000", 2024-09-21T23:38:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2784999999999997e-05 2024-09-21T23:38:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985470+0000", 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6e-06 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.985532+0000", 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2457000000000002e-05 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.102 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:23.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:23.104 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:38:23.105 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:23.105+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:23.205 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:23.202+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:23.224 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:23.224 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:23.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:23.325 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:23.326+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:23.398 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:23.399 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:23.407 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:23.406+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:23.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:23.509 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:23.506+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:23.582 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4212.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.315590056999994, 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001359416, 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267651+0000", 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267652+0000", 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.207e-06 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267670+0000", 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8539e-05 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267692+0000", 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1636000000000001e-05 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267710+0000", 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8249000000000001e-05 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267802+0000", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1751999999999998e-05 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.267853+0000", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0732999999999997e-05 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.268926+0000", 2024-09-21T23:38:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010729979999999999 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:17.269010+0000", 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4301999999999993e-05 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4646.0:2 2.0 e47/37 2:1994da04:ns::rbd_trash:head v 47'1, mlcod=0'0)", 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:19.980605+0000", 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6026358279999999, 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083921400000000002, 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980605+0000", 2024-09-21T23:38:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980605+0000", 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980605+0000", 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980613+0000", 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7929999999999998e-06 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980637+0000", 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3799999999999999e-05 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980652+0000", 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4863e-05 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980801+0000", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014866799999999999 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.980813+0000", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.287e-05 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.981414+0000", 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000600733 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:19.981444+0000", 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9980000000000001e-05 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4652.0:6 2.3 e47/17 2:daeef835:::rbd_id.testrbdnbd21555:head v 47'2, mlcod=0'0)", 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:20.042738+0000", 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.5405026579999999, 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011348949999999999, 2024-09-21T23:38:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042738+0000", 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042738+0000", 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042739+0000", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.063e-06 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042747+0000", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6009999999999997e-06 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042763+0000", 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.588e-05 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042772+0000", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7259999999999997e-06 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042828+0000", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5948e-05 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.042847+0000", 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8442000000000001e-05 2024-09-21T23:38:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.043821+0000", 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097394500000000004 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:20.043873+0000", 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2290000000000002e-05 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:23.596 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:23.597 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:23.641 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:23.642+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:23.710 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:23.710+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:23.732 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:23.732 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:23.732 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:23.732 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:23.746 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:23.811 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:23.810+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:23.862 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:23.863 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:23.863 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:23.863 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:23.863 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:23.878 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:23.927 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:23.929+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:24.002 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:24.002 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:24.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:24.002 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:24.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.773556541999994, 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.003 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:24.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:24.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:24.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:24.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.004371086999996, 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.009 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.004239564000002, 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.015 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:24.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.020 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.004179655000002, 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.021 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:24.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:24.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:24.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.903352378999998, 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.027 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:24.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:24.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:24.032 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.903236890000002, 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:24.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:24.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:24.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:24.038 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736965177000002, 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:24.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:24.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.044 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736918266999993, 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:24.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:24.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736876881000001, 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:24.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:24.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:24.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736700009000003, 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:24.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:24.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736652312999993, 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.063 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.068 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736292129000006, 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:24.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.074 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736164173000006, 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:24.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:24.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.736126897000005, 2024-09-21T23:38:24.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:24.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:24.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:24.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.372367491999995, 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:24.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:24.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.372332732000004, 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.092 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:24.097 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.372236806000004, 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:24.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:24.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.103 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.757515355999999, 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.104 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:24.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:24.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.757456500000004, 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:24.110 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:24.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.757343306999999, 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.116 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:24.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:24.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:24.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:24.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.122 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:24.122 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:24.122 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:24.125 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:38:24.125 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.961095 2024-09-21T23:38:24.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.9610947003674636 2024-09-21T23:38:24.126 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd-nbd 2024-09-21T23:38:24.127 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd-nbd 2024-09-21T23:38:24.127 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:24.128 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:24.125+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:24.224 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:24.222+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:24.237 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: must specify command 2024-09-21T23:38:24.239 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.239 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd-nbd INVALIDCMD 2024-09-21T23:38:24.239 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd-nbd INVALIDCMD 2024-09-21T23:38:24.267 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: unknown command: INVALIDCMD 2024-09-21T23:38:24.268 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.268 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:24.270 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -ne 0 ']' 2024-09-21T23:38:24.270 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:24.270 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:24.304 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:24.304 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:24.304 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:24.304 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:24.318 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:24.325 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:24.322+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:24.373 INFO:tasks.workunit.client.0.smithi165.stderr:modprobe: ERROR: could not insert 'nbd': Operation not permitted 2024-09-21T23:38:24.377 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: Netlink interface not supported. Using ioctl interface. 2024-09-21T23:38:24.377 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: failed to find unused device 2024-09-21T23:38:24.385 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: rbd-nbd failed with error: /usr/bin/rbd-nbd: exit status: 1 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo rbd device --device-type nbd map INVALIDIMAGE 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd map INVALIDIMAGE 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:24.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:24.390 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:24.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:24.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:24.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd map INVALIDIMAGE 2024-09-21T23:38:24.426 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:24.426+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:24.438 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:24.439 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:24.439 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:24.439 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:24.439 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:24.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:24.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:24.481 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: rbd-nbd failed with error: /usr/bin/rbd-nbd: exit status: 1 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo rbd-nbd --device INVALIDDEV map testrbdnbd21555 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd-nbd --device INVALIDDEV map testrbdnbd21555 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:24.485 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:24.486 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd-nbd 2024-09-21T23:38:24.487 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd-nbd 2024-09-21T23:38:24.487 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:24.487 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd-nbd --device INVALIDDEV map testrbdnbd21555 2024-09-21T23:38:24.527 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:24.526+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:24.557 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: invalid device path: INVALIDDEV (expected /dev/nbd{num}) 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:8 2.7 e47/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 47'1, mlcod=0'0)", 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.5199517580000004, 2024-09-21T23:38:24.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001699575, 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049677+0000", 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049698+0000", 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1169e-05 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049722+0000", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3912000000000001e-05 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049740+0000", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8428999999999999e-05 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049876+0000", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013545000000000001 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049935+0000", 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9657000000000002e-05 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051278+0000", 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001342389 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051375+0000", 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7355999999999994e-05 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.570 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:21 2.7 e48/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 48'2, mlcod=0'0)", 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.4957022779999996, 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001378446, 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073933+0000", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.977e-06 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073960+0000", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6435e-05 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073975+0000", 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4905999999999999e-05 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.074119+0000", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000143943 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.074198+0000", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9649999999999998e-05 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.075233+0000", 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001034828 2024-09-21T23:38:24.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.075303+0000", 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0197000000000006e-05 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:23 2.7 e48/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 48'3, mlcod=47'1)", 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.4866791490000004, 2024-09-21T23:38:24.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001385261, 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082950+0000", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7430000000000001e-06 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082961+0000", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1732999999999999e-05 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082991+0000", 2024-09-21T23:38:24.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9802e-05 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083005+0000", 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3502e-05 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083171+0000", 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000166236 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083228+0000", 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7163000000000001e-05 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.084241+0000", 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010129029999999999 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.084333+0000", 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2179e-05 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4579.0:12 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'11, mlcod=48'9)", 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.20508462799999999, 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082638799999999995, 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:24.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364544+0000", 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5960000000000001e-06 2024-09-21T23:38:24.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364556+0000", 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1986e-05 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364580+0000", 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3652e-05 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364591+0000", 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1674999999999999e-05 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364663+0000", 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1812999999999998e-05 2024-09-21T23:38:24.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364689+0000", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6123999999999999e-05 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.365321+0000", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00063193099999999996 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.365369+0000", 2024-09-21T23:38:24.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611000000000002e-05 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4579.0:15 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'12, mlcod=48'10)", 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.18691004999999999, 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00091617000000000001, 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:24.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382718+0000", 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3349999999999999e-06 2024-09-21T23:38:24.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382730+0000", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1503e-05 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382753+0000", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3538999999999999e-05 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382765+0000", 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.173e-05 2024-09-21T23:38:24.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382899+0000", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013378700000000001 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382938+0000", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9405e-05 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.383590+0000", 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065128199999999999 2024-09-21T23:38:24.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.383633+0000", 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3588999999999999e-05 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:24.589 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:24.590 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:24.591 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:24.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd device --device-type nbd --format INVALID list 2024-09-21T23:38:24.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd --format INVALID list 2024-09-21T23:38:24.591 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:24.594 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: the argument for option '--format' is invalid 2024-09-21T23:38:24.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd --format json --pretty-format list 2024-09-21T23:38:24.604 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:24.604 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:24.623 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:24.623 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:24.635 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:24.637+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:24.640 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:24.640 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:24.643 INFO:tasks.workunit.client.0.smithi165.stdout:[] 2024-09-21T23:38:24.646 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd --format xml list 2024-09-21T23:38:24.658 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:24.659 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:24.678 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:24.695 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:24.695 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:24.695 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:24.696 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:24.696 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:24.697 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:24.698 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:24.698 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:24.710 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:24.711 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:24.725 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:24.737 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:24.738+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:24.828 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:24.831 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:24.830+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:24.887 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:24.887 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:24.896 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=21859 2024-09-21T23:38:24.896 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 21859 2024-09-21T23:38:24.896 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 21859 -C rbd-nbd 2024-09-21T23:38:24.899 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:24.902 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:24.902 INFO:tasks.workunit.client.0.smithi165.stdout: 21859 ? 00:00:00 rbd-nbd 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21555 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21555 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:24.903 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd-nbd 2024-09-21T23:38:24.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd-nbd 2024-09-21T23:38:24.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:24.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21555 2024-09-21T23:38:24.932 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:24.930+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:24.972 INFO:tasks.workunit.client.0.smithi165.stderr:rbd-nbd: netlink connect failed: Object busy 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ dev1=/dev/nbd0 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 21859 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=21859 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:24.981 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:24.982 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:24.982 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:24.983 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:24.983 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:24.983 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:25.024 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.024 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.032 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:25.030+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:25.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:25.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:25.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^21859\b' 2024-09-21T23:38:25.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^21859\b' 2024-09-21T23:38:25.097 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:25.097 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:25.097 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *21859\b' 2024-09-21T23:38:25.098 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *21859\b' 2024-09-21T23:38:25.100 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 0.961095 (8629862) 2024-09-21T23:38:25.103 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:25.104 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:25.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:38:25.133 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:25.133+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:25.177 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.177 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:25.237 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:25.238+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:25.335 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.337+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.436 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.437+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.488 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.488 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:25.549 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:25.550+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:25.605 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:25.605 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21555 2024-09-21T23:38:25.606 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:25.606 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:25.608 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:25.608 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd-nbd 2024-09-21T23:38:25.610 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd-nbd 2024-09-21T23:38:25.610 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:25.610 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd-nbd --device /dev/nbd0 map rbd/testrbdnbd21555 2024-09-21T23:38:25.627 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.628 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.638 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.637+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:25.716 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:25.716 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' /dev/nbd0 = /dev/nbd0 ']' 2024-09-21T23:38:25.717 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:25.717 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep testrbdnbd21555 2024-09-21T23:38:25.739 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.741+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.776 INFO:tasks.workunit.client.0.smithi165.stdout:21955 rbd testrbdnbd21555 - /dev/nbd0 1b0423c1-ab55-4b96-af11-6546732c59ce 2024-09-21T23:38:25.777 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:25.777 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:25.777 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:25.777 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:25.777 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:25.808 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.808 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:25.840 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.841+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.845 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=21955 2024-09-21T23:38:25.846 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 21955 2024-09-21T23:38:25.846 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 21955 -C rbd-nbd 2024-09-21T23:38:25.851 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:25.851 INFO:tasks.workunit.client.0.smithi165.stdout: 21955 ? 00:00:00 rbd-nbd 2024-09-21T23:38:25.852 INFO:tasks.workunit.client.0.smithi165.stderr:++ md5sum 2024-09-21T23:38:25.852 INFO:tasks.workunit.client.0.smithi165.stderr:++ dd if=/tmp/tmp.h0CUjCtN4b/data bs=1M 2024-09-21T23:38:25.939 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:25.940 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:25.940 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:25.941+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:25.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.113845 s, 589 MB/s 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:++ md5sum 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo dd if=/dev/nbd0 bs=1M 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:25.971 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which dd 2024-09-21T23:38:25.972 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/dd 2024-09-21T23:38:25.972 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:25.972 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/dd if=/dev/nbd0 bs=1M 2024-09-21T23:38:26.041 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:26.041+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:26.074 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:26.074 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:26.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:26.119 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:38:26.181 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:26.182+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:26.215 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:26.216 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:26.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:26.242 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:26.241+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:26.343 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:26.342+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:26.393 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:26.393 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:26.408 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:26.465 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:26.465+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:26.545 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:26.546+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:26.546 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:26.546 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:26.562 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:26.657 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:26.657+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:26.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:26.712 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:26.712 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:26.712 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.73315 s, 91.5 MB/s 2024-09-21T23:38:26.714 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' '898c4092e8f841bc5f74d935a96dfb64 -' = '898c4092e8f841bc5f74d935a96dfb64 -' ']' 2024-09-21T23:38:26.715 INFO:tasks.workunit.client.0.smithi165.stderr:+ dd if=/dev/urandom of=/tmp/tmp.h0CUjCtN4b/data bs=1M count=64 2024-09-21T23:38:26.736 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:26.736 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:26.747 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:26.746+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:26.751 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:26.847 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:26.846+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:26.848 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:26.848 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:26.865 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:26.865 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:26.882 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:26.882 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:26.902 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:26.902 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:26.922 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:26.922 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:26.923 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.692599422000001, 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.923413967000002, 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011445789, 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 35 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999865+0000", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999866+0000", 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999958+0000", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2415000000000002e-05 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999972+0000", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3404000000000001e-05 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999981+0000", 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9299999999999992e-06 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007196+0000", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072156649999999996 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007340+0000", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014358200000000001 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.007578+0000", 2024-09-21T23:38:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023813799999999999 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010069+0000", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024913299999999999 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011155+0000", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010860220000000001 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011236+0000", 2024-09-21T23:38:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0451999999999995e-05 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011311+0000", 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5086999999999994e-05 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.923282443999994, 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012681008000000001, 2024-09-21T23:38:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999997+0000", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.999998+0000", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000004+0000", 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000033+0000", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8943999999999999e-05 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000041+0000", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8629999999999997e-06 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008748+0000", 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087070829999999991 2024-09-21T23:38:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.008899+0000", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000151008 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.009136+0000", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023637199999999999 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011349+0000", 2024-09-21T23:38:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022131690000000001 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012531+0000", 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011819210000000001 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012613+0000", 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1913e-05 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012678+0000", 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4739000000000005e-05 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.923222534999994, 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013040143000000001, 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000056+0000", 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000057+0000", 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000064+0000", 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2469999999999997e-06 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000077+0000", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3135999999999999e-05 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.000084+0000", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010143+0000", 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059243000000001 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010388+0000", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024469800000000001 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.010631+0000", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242525 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.011654+0000", 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001023782 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.012958+0000", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00130331 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013040+0000", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2422999999999993e-05 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.013097+0000", 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6425999999999998e-05 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.822395259000004, 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012242134999999999, 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100884+0000", 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100885+0000", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.47e-07 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100961+0000", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6365000000000005e-05 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100974+0000", 2024-09-21T23:38:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186999999999999e-05 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100983+0000", 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9199999999999993e-06 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109041+0000", 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080578809999999994 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109223+0000", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018237799999999999 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.109440+0000", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021630900000000001 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112098+0000", 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026581600000000001 2024-09-21T23:38:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112969+0000", 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087136600000000005 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113057+0000", 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8083000000000004e-05 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113126+0000", 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8639000000000005e-05 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.822279769999994, 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012737942, 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 88 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.100999+0000", 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101000+0000", 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101006+0000", 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3169999999999998e-06 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101017+0000", 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0183000000000001e-05 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.101024+0000", 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7640000000000005e-06 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111142+0000", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010117802 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111272+0000", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012979199999999999 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.111534+0000", 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026215500000000003 2024-09-21T23:38:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.112753+0000", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00121872 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113609+0000", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085596700000000001 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113683+0000", 2024-09-21T23:38:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4165999999999994e-05 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.113737+0000", 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4283000000000002e-05 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.656008056999994, 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011719452999999999, 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 110 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267271+0000", 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267272+0000", 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1069999999999999e-06 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267280+0000", 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267293+0000", 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383999999999999e-05 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267302+0000", 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.274936+0000", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076342679999999996 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275058+0000", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012194899999999999 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.275247+0000", 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000189111 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278825+0000", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035782330000000001 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278853+0000", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7440000000000002e-05 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278929+0000", 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6650000000000006e-05 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.278990+0000", 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1198999999999995e-05 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655961146999999, 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012893584, 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 111 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267318+0000", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267319+0000", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6599999999999995e-07 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267324+0000", 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267338+0000", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4243999999999999e-05 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267346+0000", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0399999999999993e-06 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276319+0000", 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089727699999999997 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276439+0000", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119919 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.276622+0000", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000182855 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279096+0000", 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002474085 2024-09-21T23:38:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280077+0000", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098043900000000005 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280153+0000", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6915999999999993e-05 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280211+0000", 2024-09-21T23:38:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8010000000000002e-05 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655919761000007, 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014044308, 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 112 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267359+0000", 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267360+0000", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6700000000000003e-07 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267493+0000", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013282600000000001 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267508+0000", 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5282999999999999e-05 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267518+0000", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.713e-06 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277390+0000", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098717630000000004 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277509+0000", 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119283 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.277717+0000", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020781900000000001 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281241+0000", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035243890000000002 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281269+0000", 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8362999999999999e-05 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281344+0000", 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5082999999999999e-05 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281404+0000", 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9119000000000001e-05 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655742888999995, 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016386306999999999, 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267536+0000", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267537+0000", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5999999999999991e-07 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267543+0000", 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0270000000000001e-06 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267554+0000", 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0686000000000001e-05 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267565+0000", 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.078e-05 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279177+0000", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612117999999999 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279296+0000", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119419 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.279507+0000", 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000210478 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283693+0000", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004186571 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283778+0000", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4689000000000002e-05 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283858+0000", 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0105999999999995e-05 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283922+0000", 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4473000000000004e-05 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655695193, 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016597201999999998, 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267584+0000", 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267585+0000", 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267590+0000", 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.784e-06 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267615+0000", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4790000000000002e-05 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267630+0000", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.468e-05 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280298+0000", 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012668478 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280421+0000", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012236899999999999 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.280602+0000", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018113600000000001 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284026+0000", 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034240329999999999 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284052+0000", 2024-09-21T23:38:26.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5707e-05 2024-09-21T23:38:26.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284122+0000", 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0455000000000002e-05 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284181+0000", 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.906e-05 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655335008999998, 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017817446000000001, 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 115 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:26.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267944+0000", 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.267946+0000", 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.517e-06 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268023+0000", 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7613000000000004e-05 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268041+0000", 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7779999999999999e-05 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268051+0000", 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0475999999999999e-05 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281485+0000", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013433993 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281619+0000", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343900000000001 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.281808+0000", 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018903799999999999 2024-09-21T23:38:26.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284208+0000", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023999490000000002 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285612+0000", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001404282 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285697+0000", 2024-09-21T23:38:26.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4796000000000006e-05 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285761+0000", 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4562999999999998e-05 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655207052999998, 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018062129999999999, 2024-09-21T23:38:26.996 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268072+0000", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268073+0000", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268079+0000", 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2330000000000003e-06 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268090+0000", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268097+0000", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3470000000000001e-06 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282807+0000", 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01470978 2024-09-21T23:38:26.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.282932+0000", 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125415 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.283124+0000", 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019143900000000001 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284985+0000", 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861569 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285991+0000", 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010057219999999999 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286072+0000", 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1236000000000006e-05 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286134+0000", 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1898999999999998e-05 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.655169776999998, 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018531717, 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 117 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268109+0000", 2024-09-21T23:38:27.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268110+0000", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0699999999999996e-07 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268116+0000", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268126+0000", 2024-09-21T23:38:27.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-05 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.268133+0000", 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284254+0000", 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016120737999999999 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284376+0000", 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122306 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.284568+0000", 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192039 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.285899+0000", 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001330489 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286511+0000", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061273600000000001 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286586+0000", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4126000000000001e-05 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.286641+0000", 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5386000000000001e-05 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.291410372000001, 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010682726, 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.008 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631869+0000", 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631874+0000", 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631881+0000", 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2300000000000002e-06 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631888+0000", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.447e-06 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637685+0000", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0057970169999999998 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637738+0000", 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2599999999999998e-05 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.637826+0000", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8109000000000006e-05 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639772+0000", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001946335 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642499+0000", 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027271259999999999 2024-09-21T23:38:27.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642528+0000", 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8147e-05 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642551+0000", 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3808e-05 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.013 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.291375611999996, 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010787626, 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 140 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631903+0000", 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631904+0000", 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631976+0000", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1846000000000003e-05 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631985+0000", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1090000000000004e-06 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631990+0000", 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9609999999999996e-06 2024-09-21T23:38:27.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638656+0000", 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066658819999999997 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638728+0000", 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2225999999999996e-05 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.638806+0000", 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8036999999999998e-05 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639783+0000", 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097759999999999991 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642642+0000", 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028585210000000002 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642669+0000", 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7336000000000001e-05 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.642691+0000", 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1742e-05 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.019 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 73.291279685999996, 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011671605999999999, 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 141 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.631999+0000", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632000+0000", 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632007+0000", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8940000000000004e-06 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632012+0000", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3820000000000003e-06 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.632016+0000", 2024-09-21T23:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639319+0000", 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00730294 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639375+0000", 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6493999999999998e-05 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.639474+0000", 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8290999999999998e-05 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.640061+0000", 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058694499999999996 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643621+0000", 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035606409999999998 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643649+0000", 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7968000000000001e-05 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:13.643671+0000", 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1801e-05 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.025 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 72.676558236000005, 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011601424000000001, 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 157 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246721+0000", 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246722+0000", 2024-09-21T23:38:27.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3300000000000001e-07 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246727+0000", 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7869999999999998e-06 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246738+0000", 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246745+0000", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7430000000000002e-06 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253263+0000", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065179299999999999 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253409+0000", 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014564499999999999 2024-09-21T23:38:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.253636+0000", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022709499999999999 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258022+0000", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004385623 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258066+0000", 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-05 2024-09-21T23:38:27.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258206+0000", 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014007900000000001 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258322+0000", 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011623199999999999 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 72.676499379999996, 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011928915, 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 158 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.032 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246780+0000", 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5000000000000001e-07 2024-09-21T23:38:27.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246855+0000", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4980000000000004e-05 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246869+0000", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3186e-05 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246877+0000", 2024-09-21T23:38:27.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6000000000000007e-06 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.254950+0000", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0080728280000000006 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255120+0000", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017016199999999999 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.255317+0000", 2024-09-21T23:38:27.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019669500000000001 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258351+0000", 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030339999999999998 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258477+0000", 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012589900000000001 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258599+0000", 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012256300000000001 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258709+0000", 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109152 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 72.676386187000006, 2024-09-21T23:38:27.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012068166999999999, 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 159 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246893+0000", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246894+0000", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8000000000000004e-07 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246899+0000", 2024-09-21T23:38:27.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246910+0000", 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0273e-05 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.246922+0000", 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2649999999999999e-05 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256839+0000", 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099164839999999997 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.256996+0000", 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015711499999999999 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.257220+0000", 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223902 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258802+0000", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001582438 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258829+0000", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6829e-05 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258903+0000", 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3899999999999994e-05 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:14.258961+0000", 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8063000000000003e-05 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.043 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.044 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:27.045 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:27.046 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:27.045+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:27.046 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:27.146 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:27.146+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:27.225 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:27.226 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.239 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:27.276 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:27.276 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:27.276 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.547764 s, 123 MB/s 2024-09-21T23:38:27.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:27.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:27.277 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:27.278 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:27.279 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:27.281 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:27.282 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:27.282 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:27.283 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:27.285+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:27.347 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:27.349+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:27.372 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:27.372 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.389 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:27.479 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:27.481+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:27.550 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:27.549+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:27.567 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:27.567 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:8 2.7 e47/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 47'1, mlcod=0'0)", 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.5203276910000003, 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001699575, 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049677+0000", 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:38:27.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049698+0000", 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1169e-05 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049722+0000", 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3912000000000001e-05 2024-09-21T23:38:27.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049740+0000", 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8428999999999999e-05 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049876+0000", 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013545000000000001 2024-09-21T23:38:27.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049935+0000", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9657000000000002e-05 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051278+0000", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001342389 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051375+0000", 2024-09-21T23:38:27.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7355999999999994e-05 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:21 2.7 e48/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 48'2, mlcod=0'0)", 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4960782110000004, 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001378446, 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.573 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073925+0000", 2024-09-21T23:38:27.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073933+0000", 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.977e-06 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073960+0000", 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6435e-05 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.073975+0000", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4905999999999999e-05 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.074119+0000", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000143943 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.074198+0000", 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9649999999999998e-05 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.075233+0000", 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001034828 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.075303+0000", 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0197000000000006e-05 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.577 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:23 2.7 e48/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 48'3, mlcod=47'1)", 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4870550820000004, 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001385261, 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082948+0000", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082950+0000", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7430000000000001e-06 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082961+0000", 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1732999999999999e-05 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.082991+0000", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9802e-05 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083005+0000", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3502e-05 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083171+0000", 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000166236 2024-09-21T23:38:27.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.083228+0000", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7163000000000001e-05 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.084241+0000", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010129029999999999 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.084333+0000", 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2179e-05 2024-09-21T23:38:27.581 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4579.0:12 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'11, mlcod=48'9)", 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.2054605610000002, 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082638799999999995, 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364542+0000", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364544+0000", 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5960000000000001e-06 2024-09-21T23:38:27.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364556+0000", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1986e-05 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364580+0000", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3652e-05 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364591+0000", 2024-09-21T23:38:27.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1674999999999999e-05 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364663+0000", 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1812999999999998e-05 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.364689+0000", 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6123999999999999e-05 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.365321+0000", 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00063193099999999996 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.365369+0000", 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611000000000002e-05 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4579.0:15 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'12, mlcod=48'10)", 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:27.586 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.1872859829999998, 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00091617000000000001, 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382717+0000", 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382718+0000", 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3349999999999999e-06 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382730+0000", 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1503e-05 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382753+0000", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3538999999999999e-05 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382765+0000", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.173e-05 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382899+0000", 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013378700000000001 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.382938+0000", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9405e-05 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.383590+0000", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065128199999999999 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.383633+0000", 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3588999999999999e-05 2024-09-21T23:38:27.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4962.0:12 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'13, mlcod=52'11)", 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.550916+0000", 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0190866650000001, 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013563970000000001, 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550916+0000", 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550916+0000", 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550918+0000", 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550935+0000", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6138999999999999e-05 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550963+0000", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8796e-05 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.550978+0000", 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4508999999999999e-05 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.551122+0000", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144554 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.551163+0000", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0941e-05 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.552189+0000", 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001025392 2024-09-21T23:38:27.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.552273+0000", 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3992999999999994e-05 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4962.0:15 2.7 e52/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 52'14, mlcod=52'12)", 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:24.562797+0000", 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0072059520000001, 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011775360000000001, 2024-09-21T23:38:27.595 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562797+0000", 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562797+0000", 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562800+0000", 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562816+0000", 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6299000000000001e-05 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562854+0000", 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7625e-05 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.562882+0000", 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7835e-05 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.563019+0000", 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000137028 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.563057+0000", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.563897+0000", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000840053 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:24.563975+0000", 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7274000000000006e-05 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.599 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:586 2.7 e54/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 54'20, mlcod=52'18)", 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.297011+0000", 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.272991818, 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00073423200000000003, 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297011+0000", 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297011+0000", 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297013+0000", 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.401e-06 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297024+0000", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1044e-05 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297045+0000", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1449000000000001e-05 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297055+0000", 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0246e-05 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297122+0000", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6177e-05 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297144+0000", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2274999999999999e-05 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297706+0000", 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056229700000000001 2024-09-21T23:38:27.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.297745+0000", 2024-09-21T23:38:27.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9342999999999999e-05 2024-09-21T23:38:27.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.605 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.605 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:611 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'21, mlcod=53'19)", 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.21125058499999999, 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019780610000000001, 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358755+0000", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3700000000000002e-06 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358998+0000", 2024-09-21T23:38:27.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000243277 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359031+0000", 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2542000000000001e-05 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359045+0000", 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4543e-05 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359124+0000", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8646000000000006e-05 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359148+0000", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3915999999999999e-05 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.360668+0000", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015200120000000001 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.360731+0000", 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2755000000000005e-05 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.20648196899999999, 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.610 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:38:27.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:613 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'23, mlcod=54'21)", 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.19946973100000001, 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023431810000000002, 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370535+0000", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.646e-06 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370730+0000", 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194842 2024-09-21T23:38:27.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370766+0000", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6409000000000003e-05 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370786+0000", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9766e-05 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370892+0000", 2024-09-21T23:38:27.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105962 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370940+0000", 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8544999999999997e-05 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.372802+0000", 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861908 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.372876+0000", 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4103000000000005e-05 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.19481793, 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.623 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:615 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'25, mlcod=54'23)", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.188050312, 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022245419999999999, 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381954+0000", 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382049+0000", 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4538999999999997e-05 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382075+0000", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6746000000000001e-05 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382089+0000", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3396e-05 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382207+0000", 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117842 2024-09-21T23:38:27.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382253+0000", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6758000000000002e-05 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.384102+0000", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001848618 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.384177+0000", 2024-09-21T23:38:27.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5222999999999995e-05 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.18325170099999999, 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.628 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.632 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:617 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'27, mlcod=54'25)", 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.17658443200000001, 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001584685, 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393420+0000", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1659999999999999e-06 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393481+0000", 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1649999999999994e-05 2024-09-21T23:38:27.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393500+0000", 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9021999999999998e-05 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393511+0000", 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0524e-05 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393622+0000", 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011063199999999999 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393661+0000", 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9428999999999998e-05 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.394970+0000", 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001308556 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.395003+0000", 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3705999999999999e-05 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:618 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.17292722699999999, 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024682649999999999, 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:27.637 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:27.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397077+0000", 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9999999999999996e-07 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397362+0000", 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028513900000000001 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397377+0000", 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5158e-05 2024-09-21T23:38:27.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397384+0000", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2150000000000004e-06 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397447+0000", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3369999999999998e-05 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397470+0000", 2024-09-21T23:38:27.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2169999999999999e-05 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.399512+0000", 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020422750000000001 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.399544+0000", 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2138000000000003e-05 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:27.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:27.642 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.643 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:27.687 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:27.689+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:27.751 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:27.750+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:27.826 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:27.827 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:27.852 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:27.850+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:27.953 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:27.954+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:27.978 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:27.978 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:27.995 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:28.041 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:28.041 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:28.041 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.750856 s, 89.4 MB/s 2024-09-21T23:38:28.043 INFO:tasks.workunit.client.0.smithi165.stderr:++ dd if=/tmp/tmp.h0CUjCtN4b/data bs=1M 2024-09-21T23:38:28.043 INFO:tasks.workunit.client.0.smithi165.stderr:++ md5sum 2024-09-21T23:38:28.054 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:28.054+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:28.123 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:28.123 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0299983319999999, 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:28.124 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:38:28.129 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0159207850000005, 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:28.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:38:28.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:38:28.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0068289739999994, 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.136 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:38:28.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.141 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.9847818779999997, 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:28.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:38:28.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:38:28.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.147 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:591 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 131072~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302248+0000", 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82330239999999999, 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093927130000000005, 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.148 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 591 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302248+0000", 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302248+0000", 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302249+0000", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5600000000000004e-07 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302389+0000", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139763 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302402+0000", 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2798e-05 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302411+0000", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.081e-06 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303293+0000", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00088277099999999997 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303366+0000", 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2447000000000006e-05 2024-09-21T23:38:28.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303488+0000", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122411 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306261+0000", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027725470000000002 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.311569+0000", 2024-09-21T23:38:28.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0053081780000000002 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.311609+0000", 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0061000000000002e-05 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.311641+0000", 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1699999999999998e-05 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:592 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 262144~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82312276600000001, 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01329383, 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 592 2024-09-21T23:38:28.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302562+0000", 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343599999999999 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302572+0000", 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0066e-05 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302578+0000", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304181+0000", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016035240000000001 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304250+0000", 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8880000000000005e-05 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304345+0000", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4862000000000001e-05 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307617+0000", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032722530000000001 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315610+0000", 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0079929819999999992 2024-09-21T23:38:28.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315670+0000", 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9567000000000001e-05 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315722+0000", 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1958e-05 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:593 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 393216~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82296253900000005, 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01328609, 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 593 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:38:28.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302694+0000", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010562500000000001 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302702+0000", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302706+0000", 2024-09-21T23:38:28.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304898+0000", 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021921900000000001 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304962+0000", 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3880000000000005e-05 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305074+0000", 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011171300000000001 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307622+0000", 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025482429999999999 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315786+0000", 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081640129999999995 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315829+0000", 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3272000000000002e-05 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315874+0000", 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4733e-05 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.165 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:594 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 524288~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82283689999999998, 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013269122, 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 594 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7e-07 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302823+0000", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109274 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302830+0000", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1010000000000003e-06 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302835+0000", 2024-09-21T23:38:28.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305597+0000", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027625810000000001 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305660+0000", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2440999999999994e-05 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305750+0000", 2024-09-21T23:38:28.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9801000000000003e-05 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308358+0000", 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002608731 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315916+0000", 2024-09-21T23:38:28.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007558085 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315953+0000", 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6588000000000003e-05 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315983+0000", 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9635999999999999e-05 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:595 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 655360~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.822708781, 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013245257999999999, 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 595 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:28.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302946+0000", 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103844 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302953+0000", 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1600000000000001e-06 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302957+0000", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306301+0000", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033432940000000001 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306364+0000", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3269999999999996e-05 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306468+0000", 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010397 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308363+0000", 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018950239999999999 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316021+0000", 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076579980000000001 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316059+0000", 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7957000000000003e-05 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316087+0000", 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8038000000000002e-05 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:596 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 786432~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.177 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82258607900000003, 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014017442999999999, 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 596 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302965+0000", 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303074+0000", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108726 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303082+0000", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303087+0000", 2024-09-21T23:38:28.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306999+0000", 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003911922 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307058+0000", 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9244999999999999e-05 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307152+0000", 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3790999999999997e-05 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308367+0000", 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001215109 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316919+0000", 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085514760000000006 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316955+0000", 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6202000000000003e-05 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316982+0000", 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6965999999999999e-05 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:597 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 917504~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.183 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.82245680099999996, 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014724014000000001, 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 597 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303180+0000", 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5927000000000006e-05 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303188+0000", 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0940000000000008e-06 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303192+0000", 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3320000000000002e-06 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307647+0000", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044549960000000001 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307726+0000", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8580999999999999e-05 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307833+0000", 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106912 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308708+0000", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000874565 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317661+0000", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089531290000000006 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317753+0000", 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2410000000000005e-05 2024-09-21T23:38:28.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317818+0000", 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4629999999999996e-05 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:603 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1572864~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319559+0000", 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.805991083, 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086890279999999997, 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.189 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 603 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319559+0000", 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319559+0000", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319560+0000", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319650+0000", 2024-09-21T23:38:28.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0150999999999998e-05 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319658+0000", 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1559999999999995e-06 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319663+0000", 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3669999999999999e-06 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.324356+0000", 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046934250000000002 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.324559+0000", 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020317100000000001 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.324761+0000", 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000201919 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327483+0000", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027218199999999998 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328179+0000", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00069652699999999995 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328218+0000", 2024-09-21T23:38:28.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8583e-05 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328248+0000", 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0440999999999999e-05 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:604 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1703936~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319670+0000", 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.80588083700000002, 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010445658999999999, 2024-09-21T23:38:28.195 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 604 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319670+0000", 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319670+0000", 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319670+0000", 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0800000000000005e-07 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319793+0000", 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122657 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319799+0000", 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5690000000000001e-06 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319804+0000", 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-06 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326027+0000", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0062233699999999998 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326087+0000", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9629000000000001e-05 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326180+0000", 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3185999999999999e-05 2024-09-21T23:38:28.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327495+0000", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00131532 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330043+0000", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025479220000000002 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330078+0000", 2024-09-21T23:38:28.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330115+0000", 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7154999999999999e-05 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:605 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1835008~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319811+0000", 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.80573966699999999, 2024-09-21T23:38:28.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010396223, 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 605 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319811+0000", 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319811+0000", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319811+0000", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319894+0000", 2024-09-21T23:38:28.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2447999999999994e-05 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319903+0000", 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7770000000000008e-06 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319907+0000", 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2010000000000001e-06 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326733+0000", 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068259640000000003 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326792+0000", 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9020000000000001e-05 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.326881+0000", 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9310999999999999e-05 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328260+0000", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013791809999999999 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330146+0000", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018855899999999999 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330181+0000", 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5253999999999998e-05 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330207+0000", 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5993999999999999e-05 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:606 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1966080~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.80563719499999997, 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010800300000000001, 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.207 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 606 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319914+0000", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320014+0000", 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100796 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320023+0000", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320029+0000", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327520+0000", 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074910100000000002 2024-09-21T23:38:28.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327580+0000", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9895000000000002e-05 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327669+0000", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9441e-05 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328726+0000", 2024-09-21T23:38:28.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010565749999999999 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330656+0000", 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019300999999999999 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330690+0000", 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4329000000000003e-05 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330714+0000", 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3623000000000001e-05 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:694 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.835057+0000", 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.29049354500000002, 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081128599999999995, 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.213 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 694 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835057+0000", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835057+0000", 2024-09-21T23:38:28.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835059+0000", 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.612e-06 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835953+0000", 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089445499999999995 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835982+0000", 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9379000000000001e-05 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.835998+0000", 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5129e-05 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.836151+0000", 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015388399999999999 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.836405+0000", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025333300000000001 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.836780+0000", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00037490100000000002 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.841757+0000", 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049770049999999996 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.843007+0000", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001250475 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.843096+0000", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9153000000000005e-05 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.843170+0000", 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3534000000000004e-05 2024-09-21T23:38:28.218 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:696 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.848199+0000", 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.27735115900000001, 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083610609999999995, 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 696 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.848199+0000", 2024-09-21T23:38:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.848199+0000", 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.848200+0000", 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.004e-06 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849202+0000", 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00100197 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849234+0000", 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1426e-05 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849249+0000", 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5333e-05 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849331+0000", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1914000000000002e-05 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849433+0000", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010186100000000001 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.849541+0000", 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010793000000000001 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.851677+0000", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021362199999999999 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.856473+0000", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047963290000000002 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.856521+0000", 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7791999999999999e-05 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.856560+0000", 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9282000000000001e-05 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:716 2.3 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.947333+0000", 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.17821725499999999, 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081894840000000003, 2024-09-21T23:38:28.226 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 716 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.947333+0000", 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.947333+0000", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.947334+0000", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7199999999999997e-07 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948178+0000", 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00084406900000000003 2024-09-21T23:38:28.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948190+0000", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2173e-05 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948196+0000", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.575e-06 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948255+0000", 2024-09-21T23:38:28.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9236e-05 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948352+0000", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6595999999999995e-05 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.948493+0000", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014114499999999999 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.951101+0000", 2024-09-21T23:38:28.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026076350000000001 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.955449+0000", 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043480899999999998 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.955492+0000", 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3528000000000001e-05 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.955523+0000", 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0465000000000001e-05 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:718 2.3 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.960856+0000", 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.164694392, 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090490290000000001, 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.232 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 718 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.960856+0000", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.960856+0000", 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.960857+0000", 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0640000000000001e-06 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.961799+0000", 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00094186200000000002 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.961812+0000", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.308e-05 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.961821+0000", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2709999999999998e-06 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.961880+0000", 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8653000000000002e-05 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.961971+0000", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0684000000000001e-05 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.962102+0000", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131707 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.964708+0000", 2024-09-21T23:38:28.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026057430000000002 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.969797+0000", 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005088581 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.969856+0000", 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9024999999999998e-05 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.969905+0000", 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9359e-05 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:728 2.3 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head [write 2228224~917504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.022902+0000", 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.102648277, 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094981130000000007, 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:28.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 728 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.022902+0000", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:28.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.022902+0000", 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.022903+0000", 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.23e-07 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024046+0000", 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001142774 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024057+0000", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1321999999999999e-05 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024063+0000", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.173e-06 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024117+0000", 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3687000000000003e-05 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024211+0000", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4511000000000004e-05 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.024352+0000", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014012700000000001 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.026426+0000", 2024-09-21T23:38:28.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074677 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.032253+0000", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0058267620000000001 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.032338+0000", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4824e-05 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.032400+0000", 2024-09-21T23:38:28.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2533000000000006e-05 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:28.244 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:28.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:38:28.246 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:28.246 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:28.246 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.135404 s, 496 MB/s 2024-09-21T23:38:28.246 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --no-progress export testrbdnbd21555 - 2024-09-21T23:38:28.246 INFO:tasks.workunit.client.0.smithi165.stderr:++ md5sum 2024-09-21T23:38:28.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:28.247 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:28.246+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:28.269 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:38:28.268+0000 7fd20c744640 0 -- 172.21.15.165:0/320471779 >> [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] conn(0x55c9fbc59ff0 msgr2=0x55c9fbc5c3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:38:28.302 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:38:28.300+0000 7fd20b4e2640 0 -- 172.21.15.165:0/320471779 >> [v2:172.21.15.118:6808/1410364236,v1:172.21.15.118:6809/1410364236] conn(0x7fd1f0008c60 msgr2=0x7fd1f000b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:38:28.346 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:28.349+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:28.374 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:28.374 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:28.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' '9dcc1be34d981527d64ce55c9f9bc889 -' = '9dcc1be34d981527d64ce55c9f9bc889 -' ']' 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 21955 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=21955 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:28.427 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:28.428 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:28.428 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:28.428 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:28.447 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:28.449+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:28.506 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:28.506 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:28.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:28.521 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:28.521 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^21955\b' 2024-09-21T23:38:28.521 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^21955\b' 2024-09-21T23:38:28.548 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:28.549+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:28.575 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:28.575 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:28.575 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *21955\b' 2024-09-21T23:38:28.575 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *21955\b' 2024-09-21T23:38:28.581 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:28.582 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:28.645 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:28.645 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:28.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:28.645 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:28.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.625111552999996, 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:28.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.015301206000004, 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.014593418000004, 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:28.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:28.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.014045909000004, 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:28.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.012437578000004, 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:28.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.011780880000003, 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:28.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:28.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:28.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.011249124000003, 2024-09-21T23:38:28.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:28.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.010435463999997, 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.009963561000006, 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.681 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:28.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:28.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.008933928999994, 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:28.685 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:28.689 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.008311280000001, 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:28.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.693 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.96601880799999995, 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.95863868100000005, 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.94603204500000004, 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.93445571900000002, 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.92298466499999998, 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.91091465900000002, 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.89892011599999999, 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.88722296700000003, 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:28.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:28.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.87579920600000005, 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:28.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:28.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:28.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:28.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:28.733 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:28.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:28.735 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:28.734+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:28.835 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:28.837+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:28.903 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:28.903 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:28.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:28.961 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:28.962+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:29.036 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:29.037 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:29.037 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:29.034+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:29.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:29.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:29.084 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd --options notrim map rbd/testrbdnbd21555 2024-09-21T23:38:29.084 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:29.084 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:29.085 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:29.086 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:29.088 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:29.088 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:29.088 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --options notrim map rbd/testrbdnbd21555 2024-09-21T23:38:29.131 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:38:29.137 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:29.137+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:29.193 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:29.193 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:29.209 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:29.209 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:29.227 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:29.228 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:29.228 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.228 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.237 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:29.237 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:29.238 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:29.238 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:29.238 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:29.238+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:29.238 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:29.239 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:29.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:29.249 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:29.249 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:29.271 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:29.271 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:29.291 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:29.307 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22066 2024-09-21T23:38:29.307 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22066 2024-09-21T23:38:29.307 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22066 -C rbd-nbd 2024-09-21T23:38:29.313 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:29.313 INFO:tasks.workunit.client.0.smithi165.stdout: 22066 ? 00:00:00 rbd-nbd 2024-09-21T23:38:29.314 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:29.314 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-21T23:38:29.339 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:29.337+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:29.363 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.364 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ provisioned=67108864 2024-09-21T23:38:29.372 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:29.373 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-21T23:38:29.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:29.429 INFO:tasks.workunit.client.0.smithi165.stderr:+ used=67108864 2024-09-21T23:38:29.429 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-21T23:38:29.429 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo blkdiscard /dev/nbd0 2024-09-21T23:38:29.429 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard /dev/nbd0 2024-09-21T23:38:29.430 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:29.430 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:29.430 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:29.430 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:38:29.431 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:38:29.431 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:29.431 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard /dev/nbd0 2024-09-21T23:38:29.439 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:29.441+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:29.473 INFO:tasks.workunit.client.0.smithi165.stderr:blkdiscard: /dev/nbd0: BLKDISCARD ioctl failed: Operation not supported 2024-09-21T23:38:29.474 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:29.474 INFO:tasks.workunit.client.0.smithi165.stderr:+ sync 2024-09-21T23:38:29.494 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.494 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:29.553 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:29.554+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:29.632 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.632 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.641 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:29.642+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:29.646 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:29.742 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:29.742+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:29.776 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.777 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.792 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:29.843 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:29.841+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:29.910 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:29.910 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:29.924 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:29.944 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:29.942+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:30.045 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:30.045+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:30.107 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:30.128 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:30.128 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:30.128 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T23:38:30.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T23:38:30.151 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:30.153+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:30.247 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:30.249+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:30.274 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.274 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:30.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:30.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T23:38:30.375 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:30.377+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:30.449 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:30.450+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:30.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T23:38:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:30.473 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:30.559 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:30.561+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:30.633 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.633 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:30.648 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:30.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T23:38:30.691 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:30.693+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:30.751 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:30.753+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:30.812 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.812 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:30.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T23:38:30.828 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:30.875 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:30.877+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:30.952 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:30.950+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:30.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T23:38:30.997 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:30.997 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:31.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:31.059 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:31.061+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:31.135 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:31.135 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:31.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:31.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:38:31.191 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:31.193+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:31.255 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:31.254+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:31.261 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:31.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:31.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:31.358 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:31.357+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:31.423 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:31.423 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-21T23:38:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:31.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:31.450 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:31.450 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:31.459 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:31.461+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:31.466 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:31.467 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:31.482 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:31.483 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:31.494 INFO:tasks.workunit.client.0.smithi165.stderr:+ provisioned=67108864 2024-09-21T23:38:31.494 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:31.494 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-21T23:38:31.502 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:31.502 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:31.522 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:31.522 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:31.542 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:31.542 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:31.552 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:31.552 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:31.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:31.552 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.534212917999994, 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037186910000000001, 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018452+0000", 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018454+0000", 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.776e-06 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018467+0000", 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2914e-05 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018502+0000", 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5852999999999997e-05 2024-09-21T23:38:31.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018525+0000", 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2938999999999998e-05 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018648+0000", 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012248599999999999 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.018698+0000", 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0550000000000002e-05 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022079+0000", 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033806000000000001 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.022171+0000", 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1572999999999999e-05 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.924402571000002, 2024-09-21T23:38:31.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:31.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.923694783000002, 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:31.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.568 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:133 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'91, mlcod=16'85)", 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.923147274000002, 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037518909999999998, 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629517+0000", 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629519+0000", 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629528+0000", 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629551+0000", 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:31.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629571+0000", 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0483e-05 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630314+0000", 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074280099999999996 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.630346+0000", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2257e-05 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633229+0000", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028831239999999999 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633269+0000", 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0203000000000001e-05 2024-09-21T23:38:31.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.921538943000002, 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:31.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:31.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:31.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:136 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'94, mlcod=16'87)", 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:31.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.920882245000001, 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039182490000000004, 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631783+0000", 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631784+0000", 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631792+0000", 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-21T23:38:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631809+0000", 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7555999999999999e-05 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631827+0000", 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7937000000000001e-05 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632609+0000", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078191999999999997 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632639+0000", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0187000000000001e-05 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635643+0000", 2024-09-21T23:38:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030038399999999998 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635701+0000", 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7722000000000001e-05 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.920350489, 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.919536828999995, 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.919064926000004, 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:140 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'91)", 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.918035294000006, 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035618690000000001, 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634629+0000", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634631+0000", 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634798+0000", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016727800000000001 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634823+0000", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4533e-05 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634839+0000", 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6059999999999999e-05 2024-09-21T23:38:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635797+0000", 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095840300000000003 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635842+0000", 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4953000000000001e-05 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638148+0000", 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023058219999999999 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.638191+0000", 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3166e-05 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:141 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'99, mlcod=16'91)", 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.917412644999999, 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039109419999999997, 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635252+0000", 2024-09-21T23:38:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635253+0000", 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635265+0000", 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635296+0000", 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1053000000000002e-05 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635312+0000", 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637143+0000", 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001831322 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637175+0000", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2286999999999998e-05 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639081+0000", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001905248 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.639163+0000", 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2468999999999999e-05 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.875120173, 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8677400459999998, 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8551334100000001, 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.843557084, 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8320860300000001, 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8200160240000001, 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8080214809999999, 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7963243320000002, 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7849005710000001, 2024-09-21T23:38:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:31.648 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:31.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:31.651 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:31.650+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ used=67108864 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 22066 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22066 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:31.652 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:31.653 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:31.653 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22066\b' 2024-09-21T23:38:31.653 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22066\b' 2024-09-21T23:38:31.694 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:31.695 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22066\b' 2024-09-21T23:38:31.695 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:31.695 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22066\b' 2024-09-21T23:38:31.703 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:31.703 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:31.750 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:31.750+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:31.812 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:31.813 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:31.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:31.851 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:31.853+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:31.952 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:31.953+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:31.985 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:31.986 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:31.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:32.054 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:32.053+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:32.116 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4555.0:7 3.2 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:24.107609+0000", 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0059112189999997, 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046009359999999999, 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4555", 2024-09-21T23:38:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3266872437", 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107609+0000", 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107609+0000", 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107612+0000", 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107626+0000", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3733999999999999e-05 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107649+0000", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3586000000000001e-05 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107665+0000", 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6157000000000001e-05 2024-09-21T23:38:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.107802+0000", 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136742 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.108035+0000", 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000232735 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.108302+0000", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000267018 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.110772+0000", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024697719999999999 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.112061+0000", 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012893799999999999 2024-09-21T23:38:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.112143+0000", 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2139999999999996e-05 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:24.112210+0000", 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7216000000000002e-05 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7112321530000001, 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:38:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:38:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6999728110000003, 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:38:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:624 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6891771159999998, 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054719920000000002, 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 624 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424346+0000", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425659+0000", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013134100000000001 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425692+0000", 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3374000000000003e-05 2024-09-21T23:38:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425712+0000", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-05 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425838+0000", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126709 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425983+0000", 2024-09-21T23:38:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014441200000000001 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.426190+0000", 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020753199999999999 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429353+0000", 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031630629999999998 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429600+0000", 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024618799999999999 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429747+0000", 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014790199999999999 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429815+0000", 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8028999999999995e-05 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6783636770000001, 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:38:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:38:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:628 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'9, mlcod=48'7)", 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.449288+0000", 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6642321930000001, 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046368219999999996, 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.449288+0000", 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.449288+0000", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.449290+0000", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5799999999999999e-06 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.450263+0000", 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097283299999999999 2024-09-21T23:38:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.450296+0000", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3699999999999999e-05 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.450314+0000", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7989e-05 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.450437+0000", 2024-09-21T23:38:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122376 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.450473+0000", 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6216e-05 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.453815+0000", 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033419750000000001 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.453925+0000", 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110153 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:630 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'11, mlcod=54'9)", 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.460359+0000", 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6531616160000002, 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043489499999999999, 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.460359+0000", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.460359+0000", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.460360+0000", 2024-09-21T23:38:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.443e-06 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.461275+0000", 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091451300000000004 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.461300+0000", 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5505999999999999e-05 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.461313+0000", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.254e-05 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.461436+0000", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012312900000000001 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.461474+0000", 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7836999999999997e-05 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.464618+0000", 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031444799999999998 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.464708+0000", 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9501999999999998e-05 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6322681899999996, 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:38:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6202461030000004, 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:38:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:38:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:638 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6087676159999997, 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053659859999999997, 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 638 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504755+0000", 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5339999999999998e-06 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505534+0000", 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077814699999999997 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505566+0000", 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2487000000000003e-05 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505582+0000", 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5393e-05 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505720+0000", 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505856+0000", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013566199999999999 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.506063+0000", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206675 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509744+0000", 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003680872 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509965+0000", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221179 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510054+0000", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8899000000000004e-05 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510119+0000", 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5285999999999998e-05 2024-09-21T23:38:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5989491630000003, 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:38:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:38:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5881307009999999, 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:38:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:38:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:644 2.2 e54/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 54'17, mlcod=54'15)", 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.537676+0000", 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5758445400000003, 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044179240000000002, 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.537676+0000", 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.537676+0000", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.537677+0000", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3e-06 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.538577+0000", 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090012999999999996 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.538605+0000", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7345999999999999e-05 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.538617+0000", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2547e-05 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.538729+0000", 2024-09-21T23:38:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111265 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.538767+0000", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8634000000000003e-05 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.542004+0000", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032364260000000001 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.542094+0000", 2024-09-21T23:38:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0276000000000001e-05 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:684 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3354224410000004, 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055026119999999996, 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 684 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778100+0000", 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779170+0000", 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001069576 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779203+0000", 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3170000000000003e-05 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779218+0000", 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5023e-05 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779343+0000", 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012489200000000001 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779493+0000", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014961900000000001 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779707+0000", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021400499999999999 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783148+0000", 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034413740000000001 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783280+0000", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013202900000000001 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783491+0000", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021118499999999999 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783601+0000", 2024-09-21T23:38:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109536 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:686 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.788540+0000", 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3249801530000003, 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053069040000000003, 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 686 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.788540+0000", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.788540+0000", 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.788543+0000", 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789386+0000", 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084302699999999999 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789418+0000", 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2246000000000003e-05 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789433+0000", 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5034e-05 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789575+0000", 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141562 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789712+0000", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013752600000000001 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.789913+0000", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020132099999999999 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.793260+0000", 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033470819999999999 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.793501+0000", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000240941 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.793747+0000", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024574900000000002 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.793847+0000", 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100169 2024-09-21T23:38:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:688 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3149282820000003, 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055549559999999998, 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 688 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798594+0000", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799548+0000", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095371900000000001 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799573+0000", 2024-09-21T23:38:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-05 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799587+0000", 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3499999999999999e-05 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799692+0000", 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105036 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799825+0000", 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013321200000000001 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.800022+0000", 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019666200000000001 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803718+0000", 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003696179 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803857+0000", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013872600000000001 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804047+0000", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019052500000000001 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804147+0000", 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9943000000000002e-05 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:690 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3045115730000001, 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054051699999999999, 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 690 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809011+0000", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1600000000000001e-06 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809825+0000", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081403400000000005 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809857+0000", 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2175999999999998e-05 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809873+0000", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5207e-05 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810002+0000", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012908900000000001 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810137+0000", 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013584900000000001 2024-09-21T23:38:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810346+0000", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020878800000000001 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.813922+0000", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035753099999999999 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814169+0000", 2024-09-21T23:38:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024718899999999998 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814314+0000", 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014533800000000001 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814414+0000", 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010003 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8710201400000002, 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:38:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.870927311, 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:22 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.296458+0000", 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8170626429999999, 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00470572, 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296458+0000", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296458+0000", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296461+0000", 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.898e-06 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296476+0000", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5248999999999999e-05 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296507+0000", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0752e-05 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296520+0000", 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2911999999999999e-05 2024-09-21T23:38:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296618+0000", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8200000000000002e-05 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.296750+0000", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013243700000000001 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.301164+0000", 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044132720000000002 2024-09-21T23:38:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.226 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:32.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:32.228 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:32.228 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:32.229 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:32.230 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:38:32.230 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:32.229+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:32.329 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:32.329+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:32.353 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:32.393 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:32.393 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:32.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:32.418 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22223 2024-09-21T23:38:32.418 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22223 2024-09-21T23:38:32.418 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22223 -C rbd-nbd 2024-09-21T23:38:32.425 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:32.425 INFO:tasks.workunit.client.0.smithi165.stdout: 22223 ? 00:00:00 rbd-nbd 2024-09-21T23:38:32.427 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:32.427 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-21T23:38:32.429 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:32.429+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:32.484 INFO:tasks.workunit.client.0.smithi165.stderr:+ provisioned=67108864 2024-09-21T23:38:32.485 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:32.485 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-21T23:38:32.523 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:32.523 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:32.523 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:32.524 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:32.524 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:32.530 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:32.529+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:32.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:32.546 INFO:tasks.workunit.client.0.smithi165.stderr:+ used=67108864 2024-09-21T23:38:32.546 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 67108864 -eq 67108864 ']' 2024-09-21T23:38:32.546 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard /dev/nbd0 2024-09-21T23:38:32.546 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:32.546 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:32.547 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:32.547 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:38:32.548 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:38:32.548 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:32.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard /dev/nbd0 2024-09-21T23:38:32.631 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:32.633+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:32.650 INFO:tasks.workunit.client.0.smithi165.stderr:+ sync 2024-09-21T23:38:32.655 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:590 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'6, mlcod=48'4)", 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.300151+0000", 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3556549049999997, 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053086569999999996, 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300151+0000", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300151+0000", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300153+0000", 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.024e-06 2024-09-21T23:38:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300475+0000", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032203100000000002 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300507+0000", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1726000000000001e-05 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300523+0000", 2024-09-21T23:38:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6500000000000001e-05 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300660+0000", 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013683900000000001 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.300698+0000", 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8285999999999999e-05 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305351+0000", 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046530620000000003 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305459+0000", 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108189 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3546763679999998, 2024-09-21T23:38:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3538127949999996, 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3531502839999998, 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:38:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3524762150000003, 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:38:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:38:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3517496339999999, 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:38:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.351024013, 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3502870409999996, 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:38:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:610 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'25, mlcod=54'23)", 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.348077+0000", 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3077290939999999, 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052573139999999999, 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348077+0000", 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348077+0000", 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348078+0000", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.624e-06 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348910+0000", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083151299999999998 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348940+0000", 2024-09-21T23:38:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9856e-05 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.348953+0000", 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3637e-05 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.349072+0000", 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119126 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.349131+0000", 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8919999999999998e-05 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.353218+0000", 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040868320000000003 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.353334+0000", 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115806 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:652 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'27, mlcod=54'25)", 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0725387260000003, 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063874020000000004, 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583268+0000", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584341+0000", 2024-09-21T23:38:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072999 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584369+0000", 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7112999999999999e-05 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584382+0000", 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3672999999999999e-05 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584481+0000", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8655999999999999e-05 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584516+0000", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589566+0000", 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050499890000000004 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589654+0000", 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8598999999999997e-05 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:654 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'29, mlcod=54'27)", 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0584194450000002, 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005678677, 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597388+0000", 2024-09-21T23:38:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.72e-06 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598151+0000", 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076301400000000001 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598181+0000", 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9972999999999999e-05 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598195+0000", 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4059999999999999e-05 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598345+0000", 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015011199999999999 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598411+0000", 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5560000000000002e-05 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.602957+0000", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045466150000000004 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.603065+0000", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107623 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:656 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'31, mlcod=54'29)", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0456626910000004, 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062273229999999999, 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610145+0000", 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610983+0000", 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083891 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611010+0000", 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6927000000000002e-05 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611038+0000", 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7915999999999998e-05 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611167+0000", 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128906 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611226+0000", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8423e-05 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616286+0000", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050602449999999997 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616370+0000", 2024-09-21T23:38:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4486000000000004e-05 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:660 2.3 e54/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 54'14, mlcod=48'12)", 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.635645+0000", 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0201608020000004, 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005450135, 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.635645+0000", 2024-09-21T23:38:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.635645+0000", 2024-09-21T23:38:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.635647+0000", 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6559999999999999e-06 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.636408+0000", 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076096099999999999 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.636437+0000", 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312999999999999e-05 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.636450+0000", 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3366999999999999e-05 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.636544+0000", 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3973000000000003e-05 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.636581+0000", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6479000000000001e-05 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.641013+0000", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044319770000000001 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.641095+0000", 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2409000000000003e-05 2024-09-21T23:38:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:664 2.3 e54/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 54'18, mlcod=54'16)", 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.658823+0000", 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.9969826990000001, 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054954110000000004, 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.658823+0000", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.658823+0000", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.658825+0000", 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.626e-06 2024-09-21T23:38:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.659647+0000", 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082196099999999996 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.659676+0000", 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9425999999999999e-05 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.659689+0000", 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.332e-05 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.659785+0000", 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5738999999999999e-05 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.659822+0000", 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6633e-05 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.664239+0000", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044177699999999997 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.664318+0000", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8936000000000005e-05 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:694 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.821338055, 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056719939999999996, 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834469+0000", 2024-09-21T23:38:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-06 2024-09-21T23:38:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835363+0000", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089360600000000002 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835404+0000", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0630000000000002e-05 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835424+0000", 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0123e-05 2024-09-21T23:38:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835590+0000", 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000166625 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835656+0000", 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5510000000000001e-05 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840062+0000", 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044064239999999999 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840140+0000", 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7279000000000003e-05 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:696 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8086722960000001, 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065145940000000003, 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847135+0000", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848041+0000", 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090629299999999996 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848068+0000", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7453e-05 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848082+0000", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3407000000000001e-05 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848168+0000", 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6229000000000004e-05 2024-09-21T23:38:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848203+0000", 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4480000000000002e-05 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853578+0000", 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053756660000000003 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853648+0000", 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9733000000000005e-05 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7096570790000003, 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6958835030000001, 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:726 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'40, mlcod=55'38)", 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.008682+0000", 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6471238420000001, 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051159609999999996, 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.008682+0000", 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.008682+0000", 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.008683+0000", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.629e-06 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.009466+0000", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078263499999999995 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.009495+0000", 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8873e-05 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.009509+0000", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3866000000000001e-05 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.009639+0000", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012987900000000001 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.009698+0000", 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9426999999999999e-05 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.013715+0000", 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040170700000000002 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.013798+0000", 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2582000000000003e-05 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6337051000000002, 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:38:32.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:38:32.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:32.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:38:32.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:32.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:32.752 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:32.753 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:32.753 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:32.753+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:32.817 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:32.818 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/provisioned_size -v . 2024-09-21T23:38:32.854 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:32.853+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:32.880 INFO:tasks.workunit.client.0.smithi165.stderr:+ provisioned=67108864 2024-09-21T23:38:32.880 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd -p rbd --format xml du testrbdnbd21555 2024-09-21T23:38:32.880 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -m //stats/images/image/used_size -v . 2024-09-21T23:38:32.887 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:32.887 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:32.887 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:32.901 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ used=0 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 0 -lt 67108864 ']' 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 22223 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22223 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:32.939 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:32.940 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:32.940 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:32.941 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:32.941 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:32.941 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:32.955 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:32.953+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:33.022 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:33.022 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:33.023 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:33.023 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:33.023 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:33.024 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:33.025 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22223\b' 2024-09-21T23:38:33.025 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22223\b' 2024-09-21T23:38:33.037 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:33.057 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:33.053+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:33.072 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:33.073 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:33.073 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22223\b' 2024-09-21T23:38:33.073 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22223\b' 2024-09-21T23:38:33.078 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:33.079 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:33.157 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:33.157+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.934202560000003, 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.165 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:33.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.58689731700000003, 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:33.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:33.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.177 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.58681788400000001, 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.181 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.58678688099999998, 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:33.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:33.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.58674232299999995, 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.186 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:33.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:33.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.58670751799999998, 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.190 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:33.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56309429499999997, 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:33.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:33.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.199 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56305937900000003, 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:33.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:33.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:33.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56302763, 2024-09-21T23:38:33.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:33.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:33.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56298796900000003, 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.211 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:33.216 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56295673000000002, 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.217 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:33.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:33.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56250235299999995, 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:33.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:33.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.56212816899999996, 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:33.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:33.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.234 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.55987076000000002, 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:33.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.240 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.55552707499999998, 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:33.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:33.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:33.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.54936345200000003, 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:33.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:33.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:33.252 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.54613355699999999, 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:33.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:33.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:33.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:33.258 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.54574730100000002, 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:33.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:33.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:33.264 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.53598592099999998, 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:33.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.53575666399999999, 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.270 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:33.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:33.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.276 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:33.277 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:33.278 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:33.277+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:33.378 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:33.377+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:33.446 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:33.447 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:33.447 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:33.447 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:33.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:33.511 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:33.513+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:33.580 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:33.581+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:33.580 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device -t nbd -o try-netlink map rbd/testrbdnbd21555 2024-09-21T23:38:33.581 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:33.581 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:33.582 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:33.582 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:33.583 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:33.583 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:33.583 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:33.583 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:33.583 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:33.585 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:33.585 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:33.585 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device -t nbd -o try-netlink map rbd/testrbdnbd21555 2024-09-21T23:38:33.597 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:33.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:33.683 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:33.685+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:33.744 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4652.0:8 2.7 e47/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 47'1, mlcod=0'0)", 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.721237352999999, 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001699575, 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049675+0000", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049677+0000", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2130000000000001e-06 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049698+0000", 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1169e-05 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049722+0000", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3912000000000001e-05 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049740+0000", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8428999999999999e-05 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049876+0000", 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013545000000000001 2024-09-21T23:38:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.049935+0000", 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9657000000000002e-05 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051278+0000", 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001342389 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.051375+0000", 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7355999999999994e-05 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:611 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'21, mlcod=53'19)", 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.4121602470000001, 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019780610000000001, 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358752+0000", 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358755+0000", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3700000000000002e-06 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.358998+0000", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000243277 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359031+0000", 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2542000000000001e-05 2024-09-21T23:38:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359045+0000", 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4543e-05 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359124+0000", 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8646000000000006e-05 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.359148+0000", 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3915999999999999e-05 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.360668+0000", 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015200120000000001 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.360731+0000", 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2755000000000005e-05 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.4073916310000003, 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:38:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:613 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'23, mlcod=54'21)", 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.4003793929999997, 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023431810000000002, 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370533+0000", 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370535+0000", 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.646e-06 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370730+0000", 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194842 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370766+0000", 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6409000000000003e-05 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370786+0000", 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9766e-05 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370892+0000", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105962 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.370940+0000", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8544999999999997e-05 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.372802+0000", 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001861908 2024-09-21T23:38:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.372876+0000", 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4103000000000005e-05 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.3957275920000001, 2024-09-21T23:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:38:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:615 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'25, mlcod=54'23)", 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.3889599739999996, 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022245419999999999, 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381953+0000", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.381954+0000", 2024-09-21T23:38:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382049+0000", 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4538999999999997e-05 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382075+0000", 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6746000000000001e-05 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382089+0000", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3396e-05 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382207+0000", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117842 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.382253+0000", 2024-09-21T23:38:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6758000000000002e-05 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.384102+0000", 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001848618 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.384177+0000", 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5222999999999995e-05 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.3841613629999996, 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:38:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:617 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'27, mlcod=54'25)", 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.3774940940000002, 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001584685, 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393419+0000", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393420+0000", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1659999999999999e-06 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393481+0000", 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1649999999999994e-05 2024-09-21T23:38:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393500+0000", 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9021999999999998e-05 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393511+0000", 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0524e-05 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393622+0000", 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011063199999999999 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.393661+0000", 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9428999999999998e-05 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.394970+0000", 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001308556 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.395003+0000", 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3705999999999999e-05 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:618 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.3738368889999997, 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024682649999999999, 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397076+0000", 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397077+0000", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9999999999999996e-07 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397362+0000", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028513900000000001 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397377+0000", 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5158e-05 2024-09-21T23:38:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397384+0000", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2150000000000004e-06 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397447+0000", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3369999999999998e-05 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.397470+0000", 2024-09-21T23:38:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2169999999999999e-05 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.399512+0000", 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020422750000000001 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.399544+0000", 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2138000000000003e-05 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8982238269999998, 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:38:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:702 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8901948649999998, 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002866164, 2024-09-21T23:38:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880719+0000", 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881020+0000", 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000301341 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881038+0000", 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7932e-05 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881049+0000", 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0523000000000001e-05 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881122+0000", 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3595000000000003e-05 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881153+0000", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0102000000000001e-05 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883546+0000", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023938890000000002 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883584+0000", 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7540999999999999e-05 2024-09-21T23:38:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:704 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'34, mlcod=55'32)", 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8821586769999996, 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027588729999999998, 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888755+0000", 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.093e-06 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889090+0000", 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033504800000000001 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889109+0000", 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8627999999999999e-05 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889119+0000", 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7219999999999994e-06 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889189+0000", 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0308999999999994e-05 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889221+0000", 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1755999999999999e-05 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891424+0000", 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022030650000000001 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891513+0000", 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9252000000000005e-05 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.87431371, 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:38:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8658226490000001, 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:38:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:710 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'40, mlcod=55'38)", 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8552827369999996, 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002832621, 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915631+0000", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.158e-06 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916242+0000", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061049200000000002 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916260+0000", 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7938e-05 2024-09-21T23:38:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916268+0000", 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5590000000000006e-06 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916328+0000", 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0189999999999998e-05 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916350+0000", 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2201e-05 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918425+0000", 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074944 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918463+0000", 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7138999999999998e-05 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8466870650000002, 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:33.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:38:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:38:33.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8372827449999996, 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:38:33.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:38:33.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.191847619, 2024-09-21T23:38:33.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:38:33.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:38:33.853 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.160193236, 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:38:33.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:38:33.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:38:33.858 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.1564901619999999, 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:33.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:38:33.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:38:33.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:33.862 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:33.864 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:33.864 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22379 2024-09-21T23:38:33.864 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22379 2024-09-21T23:38:33.864 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22379 -C rbd-nbd 2024-09-21T23:38:33.864 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:33.865 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:33.865 INFO:tasks.workunit.client.0.smithi165.stdout: 22379 ? 00:00:00 rbd-nbd 2024-09-21T23:38:33.865 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:33.866 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:33.866 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:33.867 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:33.867 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:33.867 INFO:tasks.workunit.client.0.smithi165.stderr:++ basename /dev/nbd0 2024-09-21T23:38:33.867 INFO:tasks.workunit.client.0.smithi165.stderr:+ devname=nbd0 2024-09-21T23:38:33.867 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-21T23:38:33.867 INFO:tasks.workunit.client.0.smithi165.stderr:+ blocks=65536 2024-09-21T23:38:33.867 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 65536 2024-09-21T23:38:33.868 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd resize rbd/testrbdnbd21555 --size 128M 2024-09-21T23:38:33.868 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:33.868 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:33.868 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:33.861+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:33.878 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:33.878 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:33.894 INFO:tasks.workunit.client.0.smithi165.stderr: Resizing image: 100% complete...done. 2024-09-21T23:38:33.900 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:33.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:33.910 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd info rbd/testrbdnbd21555 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout:rbd image 'testrbdnbd21555': 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: size 128 MiB in 32 objects 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: order 22 (4 MiB objects) 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: snapshot_count: 0 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: id: 122ca41dcc05 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: block_name_prefix: rbd_data.122ca41dcc05 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: format: 2 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: op_features: 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: flags: 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: create_timestamp: Sat Sep 21 23:38:20 2024 2024-09-21T23:38:33.958 INFO:tasks.workunit.client.0.smithi165.stdout: access_timestamp: Sat Sep 21 23:38:20 2024 2024-09-21T23:38:33.959 INFO:tasks.workunit.client.0.smithi165.stdout: modify_timestamp: Sat Sep 21 23:38:20 2024 2024-09-21T23:38:33.962 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-21T23:38:33.964 INFO:tasks.workunit.client.0.smithi165.stderr:+ blocks2=131072 2024-09-21T23:38:33.964 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 131072 2024-09-21T23:38:33.964 INFO:tasks.workunit.client.0.smithi165.stderr:+ test 131072 -eq 131072 2024-09-21T23:38:33.964 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd resize rbd/testrbdnbd21555 --allow-shrink --size 64M 2024-09-21T23:38:33.964 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:33.965+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:34.031 INFO:tasks.workunit.client.0.smithi165.stderr: Resizing image: 50% complete... Resizing image: 53% complete... Resizing image: 56% complete... Resizing image: 59% complete... Resizing image: 62% complete... Resizing image: 65% complete... Resizing image: 68% complete... Resizing image: 71% complete... Resizing image: 75% complete... Resizing image: 78% complete... Resizing image: 81% complete... Resizing image: 84% complete... Resizing image: 87% complete... Resizing image: 90% complete... Resizing image: 93% complete... Resizing image: 96% complete... Resizing image: 100% complete...done. 2024-09-21T23:38:34.041 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.041 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.048 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk -v dev=nbd0 '$4 == dev {print $3}' /proc/partitions 2024-09-21T23:38:34.050 INFO:tasks.workunit.client.0.smithi165.stderr:+ blocks2=65536 2024-09-21T23:38:34.050 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 65536 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ test 65536 -eq 65536 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 22379 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22379 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:34.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:34.052 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:34.054 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:34.054 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:34.054 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:34.057 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:34.099 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:34.101+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:34.139 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:34.139 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22379\b' 2024-09-21T23:38:34.139 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22379\b' 2024-09-21T23:38:34.166 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:34.165+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:34.193 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:34.193 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:34.194 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22379\b' 2024-09-21T23:38:34.194 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22379\b' 2024-09-21T23:38:34.199 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:34.200 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:34.221 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.222 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:38:34.236 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:34.279 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:34.281+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:34.367 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:34.365+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:34.401 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.401 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:34.468 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:34.469+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:34.537 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:34.569 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:34.569+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:34.667 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.667 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.670 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:34.669+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:34.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:34.701 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd --device-type nbd map --read-only rbd/testrbdnbd21555 2024-09-21T23:38:34.701 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:34.701 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:34.702 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:34.702 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:34.705 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:34.705 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:34.705 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd --device-type nbd map --read-only rbd/testrbdnbd21555 2024-09-21T23:38:34.771 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:34.769+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:34.803 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.804 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:34.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:34.855 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:34.856 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd list 2024-09-21T23:38:34.856 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk -v pool=rbd -v img=testrbdnbd21555 -v dev=/dev/nbd0 '$2 == pool && $3 == img && $5 == dev {print $1}' 2024-09-21T23:38:34.905 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:34.905+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:34.917 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22505 2024-09-21T23:38:34.917 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22505 2024-09-21T23:38:34.917 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22505 -C rbd-nbd 2024-09-21T23:38:34.923 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:34.923 INFO:tasks.workunit.client.0.smithi165.stdout: 22505 ? 00:00:00 rbd-nbd 2024-09-21T23:38:34.923 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/dev/nbd0 of=/dev/null bs=1M 2024-09-21T23:38:34.924 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:34.924 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:34.924 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:34.925 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:34.926 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:34.926 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:34.926 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/dev/nbd0 of=/dev/null bs=1M 2024-09-21T23:38:34.972 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:34.969+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:34.985 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:34.985 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:35.089 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:35.089+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:35.155 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:38:35.169 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:35.169 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.172 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:35.172 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:35.172 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.240041 s, 280 MB/s 2024-09-21T23:38:35.174 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:35.173+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:35.174 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:35.174 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:35.174 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:35.175 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:35.176 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:35.177 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:35.179 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:35.179 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:35.179 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:35.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:35.187 INFO:tasks.workunit.client.0.smithi165.stderr:/usr/bin/dd: error writing '/dev/nbd0': Operation not permitted 2024-09-21T23:38:35.187 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:38:35.187 INFO:tasks.workunit.client.0.smithi165.stderr:0+0 records out 2024-09-21T23:38:35.187 INFO:tasks.workunit.client.0.smithi165.stderr:0 bytes copied, 0.000733221 s, 0.0 kB/s 2024-09-21T23:38:35.188 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 22505 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22505 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:35.189 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:35.190 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:35.190 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:35.192 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:35.193 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:35.193 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:35.275 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:35.273+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:35.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:35.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22505\b' 2024-09-21T23:38:35.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22505\b' 2024-09-21T23:38:35.328 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:35.329 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:35.329 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22505\b' 2024-09-21T23:38:35.329 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22505\b' 2024-09-21T23:38:35.335 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:35.335 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:35.343 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:35.343 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:35.401 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:35.401+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:35.477 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:35.477+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:35.528 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:35.528 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:35.578 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:35.581+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:35.679 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:35.677+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:35.706 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:35.706 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:35.805 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:35.805+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:35.837 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd --device-type nbd map --exclusive rbd/testrbdnbd21555 2024-09-21T23:38:35.838 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:35.838 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:35.840 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:35.840 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:35.842 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:35.842 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:35.843 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd --device-type nbd map --exclusive rbd/testrbdnbd21555 2024-09-21T23:38:35.881 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:35.881+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:35.885 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:35.886 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:35.900 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:35.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:35.982 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:35.981+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:36.002 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:36.002 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:36.002 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:36.002 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:36.003 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:36.003 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:36.037 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:36.037 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:36.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:36.067 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22577 2024-09-21T23:38:36.067 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22577 2024-09-21T23:38:36.067 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22577 -C rbd-nbd 2024-09-21T23:38:36.074 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:36.075 INFO:tasks.workunit.client.0.smithi165.stdout: 22577 ? 00:00:00 rbd-nbd 2024-09-21T23:38:36.075 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:36.075 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:36.075 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:36.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:36.076 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:36.078 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:36.078 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:36.078 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd0 bs=1M oflag=direct 2024-09-21T23:38:36.083 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:36.081+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:36.087 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:36.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:36.106 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:36.106 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:36.124 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:36.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:36.143 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:36.143 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:36.161 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:36.161 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:36.167 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:36.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:36.168 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:38:36.168 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:38:36.168 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:38:36.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:38:36.179 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:36.180 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:36.184 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:36.181+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:36.234 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.234 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:36.234 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:36.234 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:36.234 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:36.251 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:36.295 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:36.297+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:36.307 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.308 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "" 2024-09-21T23:38:36.308 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:36.384 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.384 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:36.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.155665681000002, 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:36.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:36.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:36.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.8083604380000002, 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:36.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:36.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.808281005, 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.398 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:36.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.8082500019999999, 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.402 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:36.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:36.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:36.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.8082054439999999, 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:36.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:36.410 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.8081706390000001, 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.411 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:36.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7845574160000002, 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.415 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:36.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:36.420 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7845225, 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.421 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:36.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.426 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7844907509999999, 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.427 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:36.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:36.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7844510900000001, 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.433 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:36.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.438 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.784419851, 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:36.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:36.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.444 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7839654739999999, 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:36.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:36.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.450 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7835912899999999, 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:36.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7813338810000001, 2024-09-21T23:38:36.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:36.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7769901959999999, 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.462 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:36.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:36.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7708265729999999, 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.468 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:36.473 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7675966779999999, 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.474 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:36.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:36.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7672104219999998, 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.480 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:36.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:36.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.485 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7574490420000002, 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.486 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:36.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:36.491 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.7572197850000002, 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:36.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:36.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:36.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:36.496 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.497 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.498 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.498 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:36.499 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:36.500 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:36.497+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:36.600 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:36.597+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:36.632 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.632 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:147 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'88, mlcod=58'86)", 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.635609+0000", 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.000297461, 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00035376899999999998, 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635609+0000", 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635609+0000", 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635611+0000", 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7630000000000001e-06 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635691+0000", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0080999999999994e-05 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635723+0000", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2359e-05 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635741+0000", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7849000000000001e-05 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635859+0000", 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117487 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.635890+0000", 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1248000000000003e-05 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 1 2024-09-21T23:38:36.636 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:36.649 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:36.701 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:36.701+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:36.778 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.779 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:36.779 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:36.779 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:36.779 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:36.794 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:36.795 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records in 2024-09-21T23:38:36.795 INFO:tasks.workunit.client.0.smithi165.stderr:64+0 records out 2024-09-21T23:38:36.795 INFO:tasks.workunit.client.0.smithi165.stderr:67108864 bytes (67 MB, 64 MiB) copied, 0.710169 s, 94.5 MB/s 2024-09-21T23:38:36.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false timeout 10 rbd bench testrbdnbd21555 --io-type write --io-size=1024 --io-total=1024 2024-09-21T23:38:36.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ timeout 10 rbd bench testrbdnbd21555 --io-type write --io-size=1024 --io-total=1024 2024-09-21T23:38:36.801 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:36.801+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:36.857 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:38:36.851+0000 7f0e12e2d640 -1 librbd::exclusive_lock::ImageDispatch: 0x7f0df4004950 handle_acquire_lock: failed to acquire exclusive lock: (30) Read-only file system 2024-09-21T23:38:36.857 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: failed to flush: (30) Read-only file system 2024-09-21T23:38:36.858 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:38:36.851+0000 7f0e12e2d640 -1 librbd::io::AioCompletion: 0x5573dc4f0010 fail: (30) Read-only file system 2024-09-21T23:38:36.858 INFO:tasks.workunit.client.0.smithi165.stderr:bench failed: (30) Read-only file system 2024-09-21T23:38:36.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:36.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd0 22577 2024-09-21T23:38:36.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd0 2024-09-21T23:38:36.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22577 2024-09-21T23:38:36.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:36.864 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:36.864 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:36.864 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:36.864 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:36.866 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:36.866 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:36.866 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd0 2024-09-21T23:38:36.902 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:36.901+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:38:36.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.5525657450000008, 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:38:36.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:38:36.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.5409017059999996, 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:38:36.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:36.924 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.5293354770000001, 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:38:36.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0433979410000003, 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:702 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:36.933 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0353689789999994, 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002866164, 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880719+0000", 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881020+0000", 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000301341 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881038+0000", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7932e-05 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881049+0000", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0523000000000001e-05 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881122+0000", 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3595000000000003e-05 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881153+0000", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0102000000000001e-05 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883546+0000", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023938890000000002 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883584+0000", 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7540999999999999e-05 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:704 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'34, mlcod=55'32)", 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0273327909999992, 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027588729999999998, 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:36.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888755+0000", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.093e-06 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889090+0000", 2024-09-21T23:38:36.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033504800000000001 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889109+0000", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8627999999999999e-05 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889119+0000", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7219999999999994e-06 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889189+0000", 2024-09-21T23:38:36.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0308999999999994e-05 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889221+0000", 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1755999999999999e-05 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891424+0000", 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022030650000000001 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891513+0000", 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9252000000000005e-05 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0194878240000005, 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:38:36.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.946 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0109967629999996, 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:710 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'40, mlcod=55'38)", 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0004568509999991, 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002832621, 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915631+0000", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.158e-06 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916242+0000", 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061049200000000002 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916260+0000", 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7938e-05 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916268+0000", 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5590000000000006e-06 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916328+0000", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0189999999999998e-05 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916350+0000", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2201e-05 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918425+0000", 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074944 2024-09-21T23:38:36.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918463+0000", 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7138999999999998e-05 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.9918611790000007, 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.955 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:38:36.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.9824568589999991, 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:38:36.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.3370217330000003, 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.964 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:38:36.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:38:36.968 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.30536735, 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:36.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.3016642760000003, 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:38:36.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:38:36.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.74606084699999997, 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:38:36.977 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:36.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:38:36.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.28751359599999998, 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:38:36.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:38:36.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.276150338, 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:38:36.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:38:36.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.26475538100000001, 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.994 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.25514664199999998, 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:36.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:38:36.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:159 2.7 e58/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 58'99, mlcod=58'97)", 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:36.999 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.22666902999999999, 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026742770000000001, 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:37.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689419+0000", 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689800+0000", 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038097400000000002 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689818+0000", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8134999999999998e-05 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689828+0000", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7939999999999997e-06 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689902+0000", 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4750000000000001e-05 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689934+0000", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.692055+0000", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021203279999999999 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.692092+0000", 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7373999999999998e-05 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.003 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.004 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.004 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:37.006 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:37.006 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:37.005+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22577\b' 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22577\b' 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22577\b' 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22577\b' 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:37.007 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:37.107 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:37.105+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:37.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:38:37.177 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:37.177 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:37.177 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:37.177 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:37.191 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:37.225 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:37.225+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:37.308 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:37.309+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:37.315 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:37.315 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:37.315 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:37.315 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:37.315 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:37.329 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:37.409 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:37.409+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:37.498 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.402146909999999, 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:38:37.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:38:37.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:37.504 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.388069363, 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:37.505 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.378977551999998, 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.510 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:38:37.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:38:37.515 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.356930456000001, 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:37.516 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:38:37.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:38:37.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.521 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:592 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 262144~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.195271344, 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01329383, 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 592 2024-09-21T23:38:37.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302562+0000", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343599999999999 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302572+0000", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0066e-05 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302578+0000", 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304181+0000", 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016035240000000001 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304250+0000", 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8880000000000005e-05 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304345+0000", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4862000000000001e-05 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307617+0000", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032722530000000001 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315610+0000", 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0079929819999999992 2024-09-21T23:38:37.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315670+0000", 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9567000000000001e-05 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315722+0000", 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1958e-05 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:593 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 393216~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.195111117, 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01328609, 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 593 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.528 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:38:37.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302694+0000", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010562500000000001 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302702+0000", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302706+0000", 2024-09-21T23:38:37.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304898+0000", 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021921900000000001 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304962+0000", 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3880000000000005e-05 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305074+0000", 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011171300000000001 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307622+0000", 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025482429999999999 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315786+0000", 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081640129999999995 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315829+0000", 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3272000000000002e-05 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315874+0000", 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4733e-05 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:594 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 524288~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.533 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.194985478, 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013269122, 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 594 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7e-07 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302823+0000", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109274 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302830+0000", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1010000000000003e-06 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302835+0000", 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305597+0000", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027625810000000001 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305660+0000", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2440999999999994e-05 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305750+0000", 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9801000000000003e-05 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308358+0000", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002608731 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315916+0000", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007558085 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315953+0000", 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6588000000000003e-05 2024-09-21T23:38:37.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315983+0000", 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9635999999999999e-05 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:595 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 655360~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.194857359, 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013245257999999999, 2024-09-21T23:38:37.539 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 595 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302946+0000", 2024-09-21T23:38:37.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103844 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302953+0000", 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1600000000000001e-06 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302957+0000", 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306301+0000", 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033432940000000001 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306364+0000", 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3269999999999996e-05 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306468+0000", 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010397 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308363+0000", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018950239999999999 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316021+0000", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076579980000000001 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316059+0000", 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7957000000000003e-05 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316087+0000", 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8038000000000002e-05 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:596 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 786432~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.194734657, 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014017442999999999, 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.545 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 596 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302965+0000", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303074+0000", 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108726 2024-09-21T23:38:37.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303082+0000", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303087+0000", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306999+0000", 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003911922 2024-09-21T23:38:37.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307058+0000", 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9244999999999999e-05 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307152+0000", 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3790999999999997e-05 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308367+0000", 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001215109 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316919+0000", 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085514760000000006 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316955+0000", 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6202000000000003e-05 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316982+0000", 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6965999999999999e-05 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:597 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 917504~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.194605379, 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014724014000000001, 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.551 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 597 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303180+0000", 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5927000000000006e-05 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303188+0000", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0940000000000008e-06 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303192+0000", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3320000000000002e-06 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307647+0000", 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044549960000000001 2024-09-21T23:38:37.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307726+0000", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8580999999999999e-05 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307833+0000", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106912 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308708+0000", 2024-09-21T23:38:37.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000874565 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317661+0000", 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089531290000000006 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317753+0000", 2024-09-21T23:38:37.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2410000000000005e-05 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317818+0000", 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4629999999999996e-05 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:606 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1966080~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.177785773, 2024-09-21T23:38:37.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010800300000000001, 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 606 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319914+0000", 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320014+0000", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100796 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320023+0000", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320029+0000", 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:38:37.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327520+0000", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074910100000000002 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327580+0000", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9895000000000002e-05 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327669+0000", 2024-09-21T23:38:37.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9441e-05 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328726+0000", 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010565749999999999 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330656+0000", 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019300999999999999 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330690+0000", 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4329000000000003e-05 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330714+0000", 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3623000000000001e-05 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.563 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:13 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2537333069999992, 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01267458, 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 13 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243968+0000", 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-06 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243978+0000", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0339999999999999e-05 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244008+0000", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9739000000000001e-05 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244023+0000", 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4912000000000001e-05 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244207+0000", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018436700000000001 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244407+0000", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019978199999999999 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256640+0000", 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012233173999999999 2024-09-21T23:38:37.567 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2536385540000001, 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0133432, 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.568 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244062+0000", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2869999999999999e-06 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244068+0000", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6420000000000001e-06 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244089+0000", 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0256000000000001e-05 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244104+0000", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.518e-05 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017795899999999999 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244456+0000", 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000173697 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257404+0000", 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012948179000000001 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.253555124, 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.572 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:38:37.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.576 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2534259859999999, 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:38:37.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2533500330000003, 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014067705999999999, 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244350+0000", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244357+0000", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2539999999999998e-06 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244381+0000", 2024-09-21T23:38:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4172000000000002e-05 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244396+0000", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4752999999999999e-05 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244518+0000", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121882 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244724+0000", 2024-09-21T23:38:37.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000206665 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.258417+0000", 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013692559999999999 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:37.585 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2217164149999995, 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01323613, 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:37.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275985+0000", 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.469e-06 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275994+0000", 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276016+0000", 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1776e-05 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276039+0000", 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3241999999999999e-05 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276160+0000", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120152 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276339+0000", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017961299999999999 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289219+0000", 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012879534999999999 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2216215689999999, 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.590 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:38:37.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.594 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2215235329999992, 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012995227, 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276177+0000", 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276182+0000", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1529999999999996e-06 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276199+0000", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7326e-05 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276215+0000", 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276330+0000", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115412 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276583+0000", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025270700000000002 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289171+0000", 2024-09-21T23:38:37.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012587783 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:27 2.3 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.1131443319999992, 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012199553, 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:37.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 27 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384556+0000", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384566+0000", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384587+0000", 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1446000000000001e-05 2024-09-21T23:38:37.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384598+0000", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1313e-05 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391058+0000", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064594429999999996 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391323+0000", 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026509 2024-09-21T23:38:37.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.396754+0000", 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054312609999999997 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:37.603 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:37.604 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:37.605 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd bench testrbdnbd21555 --io-type write --io-size=1024 --io-total=1024 2024-09-21T23:38:37.605 INFO:tasks.workunit.client.0.smithi165.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2024-09-21T23:38:37.605 INFO:tasks.workunit.client.0.smithi165.stdout: SEC OPS OPS/SEC BYTES/SEC 2024-09-21T23:38:37.605 INFO:tasks.workunit.client.0.smithi165.stdout:elapsed: 0 ops: 1 ops/sec: 250.013 bytes/sec: 250 KiB/s 2024-09-21T23:38:37.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:37.607 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:37.608 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:37.605+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:37.706 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:37.705+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:37.729 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:37.734 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:37.734 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:37.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:37.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22689 2024-09-21T23:38:37.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22689 2024-09-21T23:38:37.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22689 -C rbd-nbd 2024-09-21T23:38:37.800 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:37.800 INFO:tasks.workunit.client.0.smithi165.stdout: 22689 ? 00:00:00 rbd-nbd 2024-09-21T23:38:37.800 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device testrbdnbd21555 22689 2024-09-21T23:38:37.800 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=testrbdnbd21555 2024-09-21T23:38:37.801 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22689 2024-09-21T23:38:37.801 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:37.801 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:37.801 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:37.802 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:37.802 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:37.804 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:37.804 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:37.804 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:37.807 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:37.805+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:37.871 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:37.871 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:37.872 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:37.872 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:37.872 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:37.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:37.889 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:37.889 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22689\b' 2024-09-21T23:38:37.889 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22689\b' 2024-09-21T23:38:37.933 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:37.933+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:37.939 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:37.940 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:37.940 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22689\b' 2024-09-21T23:38:37.940 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22689\b' 2024-09-21T23:38:37.946 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:37.946 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:38.008 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:38.005+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.384115288999993, 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.383407500999994, 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.381251660999993, 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.380063207000006, 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.379249547000001, 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.378777643999996, 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.334832891, 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.327452764, 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.314846127999999, 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.303269801999999, 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.291798748, 2024-09-21T23:38:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.279728742, 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.267734199, 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.25603705, 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.244613289, 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.58404408, 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.561961822, 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.536370598, 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.5133663939999999, 2024-09-21T23:38:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.499528312, 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:38.105 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:38.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:38.149 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:38.149+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:38.167 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:38:38.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:38.210 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:38.208+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:38.279 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:38.279 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:38.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:38.341 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:38.341+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:38.389 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:38.389 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:38.411 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:38.409+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:38.413 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:38.413 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:38.430 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:38.430 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:38.447 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:38.447 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@snap 2024-09-21T23:38:38.450 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:38.451 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:38.471 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:38.471 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:38.476 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:38.476 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:38.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:38.492 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:38.492 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:38.512 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:38.512+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:38.613 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:38.616+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:38.620 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:38.620 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:38.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:38.714 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:38.712+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:38.756 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:38.756 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:38.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:38.817 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:38.817+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:38.894 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:38.894 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:38.907 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:38.916 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:38.916+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:39.017 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:39.017+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:39.075 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.075 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.091 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:39.133 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:39.133+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:39.194 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-21T23:38:39.211 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555@snap 2024-09-21T23:38:39.211 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:39.211 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:39.212 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:39.213 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:39.214 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:39.215 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:39.215 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555@snap 2024-09-21T23:38:39.219 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:39.217+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:39.227 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.227 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:39.320 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:39.317+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:39.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:39.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:39.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:39.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:39.366 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:39.366 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.367 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.367 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:39.380 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:39.421 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:39.421+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:39.435 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22786 2024-09-21T23:38:39.435 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22786 2024-09-21T23:38:39.435 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22786 -C rbd-nbd 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stdout: 22786 ? 00:00:00 rbd-nbd 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device testrbdnbd21555@snap 22786 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=testrbdnbd21555@snap 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22786 2024-09-21T23:38:39.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21555@snap 2024-09-21T23:38:39.445 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:39.445 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:39.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:39.446 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:39.447 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:39.447 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:39.447 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21555@snap 2024-09-21T23:38:39.502 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.502 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.516 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:39.529 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:39.529 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22786\b' 2024-09-21T23:38:39.529 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22786\b' 2024-09-21T23:38:39.563 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:39.564+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:39.575 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:39.576 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:39.576 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22786\b' 2024-09-21T23:38:39.576 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22786\b' 2024-09-21T23:38:39.584 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:39.584 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:39.623 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:39.621+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:39.638 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.638 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.652 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:39.724 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:39.721+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:39.817 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.817 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:39.825 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:39.824+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:39.831 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:39.925 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:39.924+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:39.999 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:39.999 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:40.013 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:40.026 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:40.025+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:40.085 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:40.086 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd snap ls rbd/testrbdnbd21555 2024-09-21T23:38:40.086 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-21T23:38:40.126 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:40.124+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:40.143 INFO:tasks.workunit.client.0.smithi165.stderr:+ SNAPID=3 2024-09-21T23:38:40.143 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --snap-id 3 rbd/testrbdnbd21555 2024-09-21T23:38:40.144 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:40.144 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:40.145 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:40.145 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:40.146 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:40.146 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:40.146 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --snap-id 3 rbd/testrbdnbd21555 2024-09-21T23:38:40.183 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:40.184 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:40.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:38:40.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:40.239 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:40.240+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:40.288 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:40.289 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:40.289 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:40.289 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:40.289 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:40.289 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:40.328 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:40.325+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:40.356 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22885 2024-09-21T23:38:40.357 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22885 2024-09-21T23:38:40.357 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22885 -C rbd-nbd 2024-09-21T23:38:40.363 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:40.363 INFO:tasks.workunit.client.0.smithi165.stdout: 22885 ? 00:00:00 rbd-nbd 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device '--snap-id 3 testrbdnbd21555' 22885 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ local 'args=--snap-id 3 testrbdnbd21555' 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22885 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap --snap-id 3 testrbdnbd21555 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:40.364 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:40.365 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:40.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:40.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:40.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --snap-id 3 testrbdnbd21555 2024-09-21T23:38:40.369 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:40.369 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:40.383 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:40.430 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:40.428+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:40.455 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:40.455 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22885\b' 2024-09-21T23:38:40.455 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22885\b' 2024-09-21T23:38:40.500 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:40.500 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:40.502 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:40.502 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:40.502 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22885\b' 2024-09-21T23:38:40.503 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22885\b' 2024-09-21T23:38:40.509 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:40.509 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:40.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:40.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:40.555 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:40.556+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:40.631 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:40.632+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:40.652 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:40.652 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:40.668 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:40.674 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:40.674 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:40.685 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:40.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:40.705 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:40.705 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:40.726 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:40.726 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:40.732 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:40.729+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:40.746 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:40.746 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:40.805 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:40.806 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:40.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:40.861 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:40.861+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:40.935 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:40.936+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.358631078000002, 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.357923290000002, 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.355767450000002, 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.354578996000001, 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.353765335999995, 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.353293433000005, 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.309348679999999, 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.301968553, 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.289361917000001, 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.277785591000001, 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.266314537, 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.254244530999999, 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.242249987999999, 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.230552839, 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.219129078, 2024-09-21T23:38:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5585598689999998, 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5364776110000005, 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5108863870000002, 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.487882183, 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4740441009999996, 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.088 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:41.090 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:41.090 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:41.091 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:41.089+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:41.182 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:38:41.190 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:41.189+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:41.200 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-21T23:38:41.216 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.216 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:41.216 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:41.218 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:41.218 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:41.220 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:41.220 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:41.220 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.265 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:41.265 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:41.265 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:41.266 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:41.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:41.325 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:41.325+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:41.327 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:41.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd ns 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns=ns 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:41.365 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:41.392 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:41.392+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:41.406 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:41.407 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:41.407 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:41.407 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:41.407 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:41.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:41.434 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=22982 2024-09-21T23:38:41.434 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 22982 2024-09-21T23:38:41.434 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 22982 -C rbd-nbd 2024-09-21T23:38:41.441 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:41.441 INFO:tasks.workunit.client.0.smithi165.stdout: 22982 ? 00:00:00 rbd-nbd 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device rbd/ns/testrbdnbd21555@snap 22982 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=22982 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:41.442 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:41.443 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:41.443 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:41.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:41.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:41.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/ns/testrbdnbd21555@snap 2024-09-21T23:38:41.493 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:41.496+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:41.518 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:41.518 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^22982\b' 2024-09-21T23:38:41.518 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^22982\b' 2024-09-21T23:38:41.548 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.140497610000001, 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:38:41.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:38:41.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:38:41.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.129238268, 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:38:41.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:38:41.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:624 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.118442572999999, 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054719920000000002, 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 624 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424346+0000", 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425659+0000", 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013134100000000001 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425692+0000", 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3374000000000003e-05 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425712+0000", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-05 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425838+0000", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126709 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425983+0000", 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014441200000000001 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.426190+0000", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020753199999999999 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429353+0000", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031630629999999998 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429600+0000", 2024-09-21T23:38:41.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024618799999999999 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429747+0000", 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014790199999999999 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429815+0000", 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8028999999999995e-05 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.107629134, 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:38:41.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:38:41.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.061533646999999, 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:41.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.049511559999999, 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:38:41.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:638 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.038033072999999, 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053659859999999997, 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 638 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504755+0000", 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5339999999999998e-06 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505534+0000", 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077814699999999997 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505566+0000", 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2487000000000003e-05 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505582+0000", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5393e-05 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505720+0000", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505856+0000", 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013566199999999999 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.506063+0000", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206675 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509744+0000", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003680872 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509965+0000", 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221179 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510054+0000", 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8899000000000004e-05 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510119+0000", 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5285999999999998e-05 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.02821462, 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:38:41.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:38:41.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.017396158, 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:38:41.601 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.602 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:41.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:38:41.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:38:41.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:684 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.610 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.764687898, 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055026119999999996, 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 684 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778100+0000", 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779170+0000", 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001069576 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779203+0000", 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3170000000000003e-05 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779218+0000", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5023e-05 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779343+0000", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012489200000000001 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779493+0000", 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014961900000000001 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779707+0000", 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021400499999999999 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783148+0000", 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034413740000000001 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783280+0000", 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013202900000000001 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783491+0000", 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021118499999999999 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783601+0000", 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109536 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:688 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.744193739, 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055549559999999998, 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 688 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.619 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798594+0000", 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799548+0000", 2024-09-21T23:38:41.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095371900000000001 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799573+0000", 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-05 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799587+0000", 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3499999999999999e-05 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799692+0000", 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105036 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799825+0000", 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013321200000000001 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.800022+0000", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019666200000000001 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803718+0000", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003696179 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803857+0000", 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013872600000000001 2024-09-21T23:38:41.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804047+0000", 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019052500000000001 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804147+0000", 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9943000000000002e-05 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:690 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.733777030000001, 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054051699999999999, 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 690 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809011+0000", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1600000000000001e-06 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809825+0000", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081403400000000005 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809857+0000", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2175999999999998e-05 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809873+0000", 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5207e-05 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810002+0000", 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012908900000000001 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810137+0000", 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013584900000000001 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810346+0000", 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020878800000000001 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.813922+0000", 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035753099999999999 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814169+0000", 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024718899999999998 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814314+0000", 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014533800000000001 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814414+0000", 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010003 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.300285597, 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:38:41.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:38:41.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.300192768, 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:38:41.638 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:38:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9633686190000006, 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:38:41.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:38:41.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:38:41.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9422162459999992, 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:41.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:38:41.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3559854150000001, 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:38:41.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:41.662 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.2206558870000004, 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:128 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0158894480000003, 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054825550000000001, 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526898+0000", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.906e-06 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527948+0000", 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010493989999999999 2024-09-21T23:38:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527974+0000", 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6407e-05 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527988+0000", 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3720000000000001e-05 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528089+0000", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100782 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528224+0000", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135649 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528433+0000", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000208982 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.531670+0000", 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032363919999999998 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532210+0000", 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053980599999999999 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532309+0000", 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9335999999999999e-05 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532379+0000", 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0176e-05 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.9956069239999996, 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:38:41.673 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:38:41.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:38:41.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:41.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:41.679 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:41.681 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:41.681 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:41.681 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *22982\b' 2024-09-21T23:38:41.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *22982\b' 2024-09-21T23:38:41.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:41.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:41.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:41.683 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:41.680+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:41.782 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:41.781+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:41.805 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:41.806 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:41.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:41.905 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:41.905+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:41.983 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:41.981+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:41.986 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:41.986 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:41.986 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:41.986 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:41.986 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:42.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:42.079 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:42.080 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd snap ls rbd/ns/testrbdnbd21555 2024-09-21T23:38:42.080 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-21T23:38:42.084 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:42.084+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:42.144 INFO:tasks.workunit.client.0.smithi165.stderr:+ SNAPID=4 2024-09-21T23:38:42.145 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --snap-id 4 rbd/ns/testrbdnbd21555 2024-09-21T23:38:42.145 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:42.145 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:42.146 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:42.146 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:42.148 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:42.148 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:42.148 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --snap-id 4 rbd/ns/testrbdnbd21555 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.863635499000001, 2024-09-21T23:38:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.862771926000001, 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:38:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.862109415000001, 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.861435346, 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.860708765, 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:38:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.859983143999999, 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:38:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:38:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.859246172000001, 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:38:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:652 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'27, mlcod=54'25)", 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.581497857, 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063874020000000004, 2024-09-21T23:38:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583268+0000", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584341+0000", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072999 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584369+0000", 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7112999999999999e-05 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584382+0000", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3672999999999999e-05 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584481+0000", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8655999999999999e-05 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584516+0000", 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:38:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589566+0000", 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050499890000000004 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589654+0000", 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8598999999999997e-05 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:654 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'29, mlcod=54'27)", 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.567378575999999, 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005678677, 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597388+0000", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.72e-06 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598151+0000", 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076301400000000001 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598181+0000", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9972999999999999e-05 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598195+0000", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4059999999999999e-05 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598345+0000", 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015011199999999999 2024-09-21T23:38:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598411+0000", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5560000000000002e-05 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.602957+0000", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045466150000000004 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.603065+0000", 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107623 2024-09-21T23:38:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:656 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'31, mlcod=54'29)", 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.554621822, 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062273229999999999, 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610145+0000", 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610983+0000", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083891 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611010+0000", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6927000000000002e-05 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611038+0000", 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7915999999999998e-05 2024-09-21T23:38:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611167+0000", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128906 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611226+0000", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8423e-05 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616286+0000", 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050602449999999997 2024-09-21T23:38:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616370+0000", 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4486000000000004e-05 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:694 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.330297185999999, 2024-09-21T23:38:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056719939999999996, 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834469+0000", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-06 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835363+0000", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089360600000000002 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835404+0000", 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0630000000000002e-05 2024-09-21T23:38:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835424+0000", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0123e-05 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835590+0000", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000166625 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835656+0000", 2024-09-21T23:38:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5510000000000001e-05 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840062+0000", 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044064239999999999 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840140+0000", 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7279000000000003e-05 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:696 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.317631427, 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065145940000000003, 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847135+0000", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848041+0000", 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090629299999999996 2024-09-21T23:38:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848068+0000", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7453e-05 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848082+0000", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3407000000000001e-05 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848168+0000", 2024-09-21T23:38:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6229000000000004e-05 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848203+0000", 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4480000000000002e-05 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853578+0000", 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053756660000000003 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853648+0000", 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9733000000000005e-05 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.21861621, 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:38:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:38:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.204842634, 2024-09-21T23:38:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:38:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:38:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.142664230999999, 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:38:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:38:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:105 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'54, mlcod=58'52)", 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7635844379999996, 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059755880000000004, 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401182+0000", 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402099+0000", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091730900000000005 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402122+0000", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402139+0000", 2024-09-21T23:38:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6976000000000001e-05 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402233+0000", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4480000000000003e-05 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402263+0000", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9522999999999998e-05 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407054+0000", 2024-09-21T23:38:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047915930000000002 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407156+0000", 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101743 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:107 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'56, mlcod=58'54)", 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.751389047, 2024-09-21T23:38:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055358780000000002, 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413377+0000", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414193+0000", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081642699999999999 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414216+0000", 2024-09-21T23:38:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2286999999999999e-05 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414233+0000", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7119999999999999e-05 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414372+0000", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414405+0000", 2024-09-21T23:38:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3673000000000001e-05 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418791+0000", 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043853770000000002 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418912+0000", 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121025 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:137 2.3 e58/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 58'58, mlcod=58'56)", 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.5911570709999996, 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060185819999999998, 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573609+0000", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4899999999999999e-06 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574550+0000", 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094037000000000005 2024-09-21T23:38:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574573+0000", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3516999999999999e-05 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574585+0000", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1712999999999999e-05 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574693+0000", 2024-09-21T23:38:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010851199999999999 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574725+0000", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.137e-05 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579522+0000", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047972780000000003 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579626+0000", 2024-09-21T23:38:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104332 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:170 2.3 e58/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 58'72, mlcod=58'70)", 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4265070140000002, 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057334100000000004, 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738260+0000", 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7e-06 2024-09-21T23:38:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739118+0000", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085804700000000002 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739147+0000", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9343e-05 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739161+0000", 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3627e-05 2024-09-21T23:38:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739247+0000", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6779000000000004e-05 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739280+0000", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3186999999999999e-05 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743866+0000", 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045857509999999999 2024-09-21T23:38:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743991+0000", 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124976 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:175 2.3 e58/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 58'76, mlcod=58'74)", 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.400359044, 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061086780000000002, 2024-09-21T23:38:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764407+0000", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.637e-06 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765253+0000", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084597600000000002 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765283+0000", 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.915e-05 2024-09-21T23:38:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765298+0000", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5444e-05 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765398+0000", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9622000000000003e-05 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765430+0000", 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2867000000000002e-05 2024-09-21T23:38:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770401+0000", 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049709890000000003 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770514+0000", 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011299299999999999 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:42.253 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:42.255 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:42.256 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:38:42.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:38:42.256 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:42.253+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:42.296 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:42.296 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd ns 2024-09-21T23:38:42.296 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:42.296 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns=ns 2024-09-21T23:38:42.297 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:42.297 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:42.356 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:42.356+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:42.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23066 2024-09-21T23:38:42.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23066 2024-09-21T23:38:42.365 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23066 -C rbd-nbd 2024-09-21T23:38:42.370 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:42.371 INFO:tasks.workunit.client.0.smithi165.stdout: 23066 ? 00:00:00 rbd-nbd 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device '--snap-id 4 rbd/ns/testrbdnbd21555' 23066 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ local 'args=--snap-id 4 rbd/ns/testrbdnbd21555' 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23066 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap --snap-id 4 rbd/ns/testrbdnbd21555 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:42.372 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:42.374 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:42.374 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:42.374 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:42.374 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:42.374 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --snap-id 4 rbd/ns/testrbdnbd21555 2024-09-21T23:38:42.389 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:42.389 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:42.389 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:42.389 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:42.403 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:42.457 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:42.457+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:42.460 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:42.460 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23066\b' 2024-09-21T23:38:42.460 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23066\b' 2024-09-21T23:38:42.511 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:42.512 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:42.512 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23066\b' 2024-09-21T23:38:42.512 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23066\b' 2024-09-21T23:38:42.518 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:42.519 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:42.558 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:42.556+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:42.570 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:42.570 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:42.570 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:42.570 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:42.570 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:42.584 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:42.660 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:42.657+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:42.752 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:42.752 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:42.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:42.752 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.523640604999997, 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.753 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:42.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:42.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.176335362, 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:42.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:42.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:42.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:42.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.770 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.770 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:42.770 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.176255929, 2024-09-21T23:38:42.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:42.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:42.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.775 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.176224926, 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.776 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:42.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.176180368000001, 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:42.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.176145563, 2024-09-21T23:38:42.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:42.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:42.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.15253234, 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:42.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:42.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.152497424, 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:42.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:42.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:42.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:42.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.152465675, 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:42.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.152426014, 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:42.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:42.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.152394774999999, 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:42.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.151940398000001, 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:42.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:42.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:42.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.151566214000001, 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:42.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:42.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.149308805, 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:42.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:42.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.14496512, 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:42.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:42.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.138801496999999, 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:42.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.135571602000001, 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:42.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:42.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:42.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.135185346, 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:42.867 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:42.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:42.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.875 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.125423966, 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:42.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:42.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.125194709000001, 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:42.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:42.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:42.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:42.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:42.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:42.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:42.891 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:42.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:42.893 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:42.893+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:42.894 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:42.995 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:42.996+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:43.020 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:43.020 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21555 2024-09-21T23:38:43.020 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:43.021 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:43.022 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:43.022 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:43.022 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:43.022 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:43.023 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:43.023 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:43.025 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:43.025 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:43.025 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21555 2024-09-21T23:38:43.036 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:43.042 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:43.042 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:43.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:43.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:43.073 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:43.073 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:43.093 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:43.093 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:43.111 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:43.111 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:43.127 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:43.128+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:43.128 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:43.129 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:43.180 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:43.180 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd ns 2024-09-21T23:38:43.180 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:43.181 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns=ns 2024-09-21T23:38:43.181 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:43.181 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:38:43.197 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:43.197+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:43.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:38:43.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:38:43.213 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.213 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.227 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:43.251 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23128 2024-09-21T23:38:43.251 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23128 2024-09-21T23:38:43.251 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23128 -C rbd-nbd 2024-09-21T23:38:43.257 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:43.257 INFO:tasks.workunit.client.0.smithi165.stdout: 23128 ? 00:00:00 rbd-nbd 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device '--pool rbd --namespace ns --image testrbdnbd21555' 23128 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:+ local 'args=--pool rbd --namespace ns --image testrbdnbd21555' 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23128 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21555 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:43.258 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:43.259 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:43.259 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:43.260 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:43.260 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:43.260 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21555 2024-09-21T23:38:43.301 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:43.301+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:43.340 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:43.341 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23128\b' 2024-09-21T23:38:43.341 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23128\b' 2024-09-21T23:38:43.390 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:43.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:43.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23128\b' 2024-09-21T23:38:43.391 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23128\b' 2024-09-21T23:38:43.396 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.397 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.397 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:43.397 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:43.398 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:43.396+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:43.412 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:43.499 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:43.497+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:43.584 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.584 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.597 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:43.600 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:43.600+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:43.701 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:43.701+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:43.717 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.717 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.731 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:43.802 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:43.800+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:43.847 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.847 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:43.899 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21555 --snap snap 2024-09-21T23:38:43.899 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:43.899 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:43.901 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:43.902 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:43.904 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:43.904 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:43.904 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --pool rbd --namespace ns --image testrbdnbd21555 --snap snap 2024-09-21T23:38:43.905 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:43.905+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:43.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:43.984 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:43.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:44.041 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:44.041+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd ns 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns=ns 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:44.061 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:44.104 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:44.104+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:44.129 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23190 2024-09-21T23:38:44.129 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23190 2024-09-21T23:38:44.129 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23190 -C rbd-nbd 2024-09-21T23:38:44.139 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:44.139 INFO:tasks.workunit.client.0.smithi165.stdout: 23190 ? 00:00:00 rbd-nbd 2024-09-21T23:38:44.140 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device '--pool rbd --namespace ns --image testrbdnbd21555 --snap snap' 23190 2024-09-21T23:38:44.140 INFO:tasks.workunit.client.0.smithi165.stderr:+ local 'args=--pool rbd --namespace ns --image testrbdnbd21555 --snap snap' 2024-09-21T23:38:44.140 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23190 2024-09-21T23:38:44.140 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21555 --snap snap 2024-09-21T23:38:44.140 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:44.141 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap --pool rbd --namespace ns --image testrbdnbd21555 --snap snap 2024-09-21T23:38:44.159 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.160 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:44.202 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:38:44.202 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:38:44.205 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:44.204+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:44.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:38:44.230 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:44.230 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23190\b' 2024-09-21T23:38:44.230 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23190\b' 2024-09-21T23:38:44.283 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:44.284 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:44.284 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23190\b' 2024-09-21T23:38:44.284 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23190\b' 2024-09-21T23:38:44.291 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:44.291 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:44.293 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.293 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.305 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:44.304+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:44.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:44.406 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:44.405+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:44.467 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.467 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:44.521 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:44.521+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:44.608 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:44.608+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:44.635 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.635 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:44.709 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:44.709+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:44.771 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.772 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:44.792 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:44.793 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:44.793 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:44.793 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:44.794 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:44.795 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:44.797 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:44.797 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:44.797 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:44.810 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:44.808+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:44.912 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:44.909+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:44.954 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:44.954 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:44.954 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:44.954 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:44.955 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:44.955 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:44.955 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:44.955 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:44.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:45.013 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:45.012+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:45.023 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23252 2024-09-21T23:38:45.023 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23252 2024-09-21T23:38:45.023 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23252 -C rbd-nbd 2024-09-21T23:38:45.032 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:45.032 INFO:tasks.workunit.client.0.smithi165.stdout: 23252 ? 00:00:00 rbd-nbd 2024-09-21T23:38:45.033 INFO:tasks.workunit.client.0.smithi165.stderr:+ pid=23252 2024-09-21T23:38:45.033 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/ns/testrbdnbd21555 2024-09-21T23:38:45.033 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:45.033 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:45.034 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:45.034 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:45.035 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:45.035 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:45.035 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/ns/testrbdnbd21555 2024-09-21T23:38:45.114 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:45.112+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:45.138 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:45.138 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:45.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:45.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:45.164 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd1 2024-09-21T23:38:45.164 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd ns 2024-09-21T23:38:45.164 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:45.164 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns=ns 2024-09-21T23:38:45.164 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:45.165 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\''ns'\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:38:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:38:45.215 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:45.216+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:45.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:38:45.243 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23298 2024-09-21T23:38:45.243 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23298 2024-09-21T23:38:45.243 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23298 -C rbd-nbd 2024-09-21T23:38:45.249 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:45.249 INFO:tasks.workunit.client.0.smithi165.stdout: 23252 ? 00:00:00 rbd-nbd 2024-09-21T23:38:45.250 INFO:tasks.workunit.client.0.smithi165.stdout: 23298 ? 00:00:00 rbd-nbd 2024-09-21T23:38:45.250 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device rbd/ns/testrbdnbd21555 23298 2024-09-21T23:38:45.250 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=rbd/ns/testrbdnbd21555 2024-09-21T23:38:45.250 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23298 2024-09-21T23:38:45.250 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap rbd/ns/testrbdnbd21555 2024-09-21T23:38:45.251 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:45.251 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:45.251 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:45.251 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:45.252 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:45.252 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:45.252 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/ns/testrbdnbd21555 2024-09-21T23:38:45.292 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:45.292 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:45.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:45.309 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:45.316 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:45.316 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:45.317 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:45.313+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:45.327 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:45.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:45.332 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:45.339 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:45.339 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23298\b' 2024-09-21T23:38:45.339 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23298\b' 2024-09-21T23:38:45.347 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:45.347 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:45.365 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:45.366 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:45.386 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:45.386 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:45.396 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:45.396 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:45.396 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23298\b' 2024-09-21T23:38:45.396 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23298\b' 2024-09-21T23:38:45.403 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:45.403 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:45.453 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:45.453+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:45.461 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:45.461 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:45.461 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:45.461 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:45.475 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:45.518 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:45.517+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:45.597 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:45.597 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:45.598 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:45.598 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:45.598 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:45.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:45.655 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:45.656+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:45.720 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:45.717+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 93.504027074999996, 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.156721832000001, 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.156642399000001, 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.156611396000001, 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.156566838, 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.156532033, 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.13291881, 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.132883894000001, 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.132852144999999, 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:45.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:45.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.132812484, 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:45.800 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:45.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:45.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:45.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.132781245, 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:45.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:45.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:45.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.132326868, 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:45.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:45.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.131952684, 2024-09-21T23:38:45.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:45.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:45.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.129695275, 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:45.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:45.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.125351589999999, 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.841 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:45.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:45.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.119187967, 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:45.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:45.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:45.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:45.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.115958072, 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:45.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:45.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:45.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.115571815999999, 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:45.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:45.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.876 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.105810436000001, 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:45.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.105581179, 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:45.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:45.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:45.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:45.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:45.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:45.889 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:45.891 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:45.891 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:45.892+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device rbd/testrbdnbd21555 23252 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=rbd/testrbdnbd21555 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23252 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap rbd/testrbdnbd21555 2024-09-21T23:38:45.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:45.905 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:45.906 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:45.907 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:45.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:45.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:45.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap rbd/testrbdnbd21555 2024-09-21T23:38:45.991 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:45.989+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:45.999 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:45.999 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23252\b' 2024-09-21T23:38:45.999 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23252\b' 2024-09-21T23:38:46.058 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:46.058 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:46.058 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23252\b' 2024-09-21T23:38:46.059 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23252\b' 2024-09-21T23:38:46.063 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.064 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:46.064 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:46.064 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:46.065 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:46.065 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:46.078 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:46.092 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:46.089+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:46.193 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:46.192+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:46.198 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.198 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:46.198 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:46.198 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:46.198 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:46.212 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:46.218 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:38:46.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:38:46.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:38:46.297 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:46.297+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:46.385 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.385 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:46.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.022590753999999, 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:38:46.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.010926715, 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:38:46.391 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:38:46.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.999360486, 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:38:46.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.401 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.513422949999999, 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:38:46.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:702 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.505393988000002, 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002866164, 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.407 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880719+0000", 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881020+0000", 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000301341 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881038+0000", 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7932e-05 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881049+0000", 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0523000000000001e-05 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881122+0000", 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3595000000000003e-05 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881153+0000", 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0102000000000001e-05 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883546+0000", 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023938890000000002 2024-09-21T23:38:46.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883584+0000", 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7540999999999999e-05 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:704 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'34, mlcod=55'32)", 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:46.412 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.4973578, 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027588729999999998, 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888755+0000", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.093e-06 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889090+0000", 2024-09-21T23:38:46.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033504800000000001 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889109+0000", 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8627999999999999e-05 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889119+0000", 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7219999999999994e-06 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889189+0000", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0308999999999994e-05 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889221+0000", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1755999999999999e-05 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891424+0000", 2024-09-21T23:38:46.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022030650000000001 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891513+0000", 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9252000000000005e-05 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:46.417 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.489512832999999, 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:38:46.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.422 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.481021771999998, 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:38:46.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:710 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'40, mlcod=55'38)", 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.47048186, 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002832621, 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:46.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915631+0000", 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.158e-06 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916242+0000", 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061049200000000002 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916260+0000", 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7938e-05 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916268+0000", 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5590000000000006e-06 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916328+0000", 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0189999999999998e-05 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916350+0000", 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2201e-05 2024-09-21T23:38:46.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918425+0000", 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074944 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918463+0000", 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7138999999999998e-05 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.461886188000001, 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:38:46.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.437 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.452481868, 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.438 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:38:46.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:38:46.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.807046742000001, 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:46.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:38:46.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.775392359, 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:38:46.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:38:46.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.453 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.771689285000001, 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:38:46.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.216085855999999, 2024-09-21T23:38:46.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:46.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:38:46.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:38:46.464 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7575386050000006, 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:38:46.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7461753469999994, 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:38:46.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:38:46.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:38:46.474 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7347803899999992, 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.475 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:38:46.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:38:46.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7251716510000001, 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:38:46.480 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:46.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.485 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:159 2.7 e58/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 58'99, mlcod=58'97)", 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.6966940390000005, 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026742770000000001, 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689418+0000", 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689419+0000", 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689800+0000", 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038097400000000002 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689818+0000", 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8134999999999998e-05 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689828+0000", 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7939999999999997e-06 2024-09-21T23:38:46.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689902+0000", 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4750000000000001e-05 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.689934+0000", 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.692055+0000", 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021203279999999999 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.692092+0000", 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7373999999999998e-05 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.490 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.491 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:46.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:46.493 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:46.492+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:46.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' rbd = rbd ']' 2024-09-21T23:38:46.567 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:46.567 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:46.567 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:46.569 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:46.569 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:46.571 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:46.571 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:46.571 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:46.593 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:46.593+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:46.662 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.662 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:46.662 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:46.662 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:46.677 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:46.693 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:46.693+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:46.718 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:46.718 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:46.718 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:46.718 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:46.719 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:46.719 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:46.786 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23392 2024-09-21T23:38:46.787 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23392 2024-09-21T23:38:46.787 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23392 -C rbd-nbd 2024-09-21T23:38:46.792 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:46.792 INFO:tasks.workunit.client.0.smithi165.stdout: 23392 ? 00:00:00 rbd-nbd 2024-09-21T23:38:46.792 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device testrbdnbd21555 23392 2024-09-21T23:38:46.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=testrbdnbd21555 2024-09-21T23:38:46.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23392 2024-09-21T23:38:46.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:46.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:46.793 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:46.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:46.794 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:46.794 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:46.792+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:46.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:46.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:46.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:46.803 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.803 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:46.803 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:46.803 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:46.803 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:46.817 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:46.876 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:46.877 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23392\b' 2024-09-21T23:38:46.877 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23392\b' 2024-09-21T23:38:46.895 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:46.893+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:46.923 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:46.924 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:46.924 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23392\b' 2024-09-21T23:38:46.924 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23392\b' 2024-09-21T23:38:46.930 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:46.930 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:46.981 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.885810694, 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:38:46.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:38:46.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.871733147, 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:38:46.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:38:46.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.862641335999999, 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:46.994 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:38:46.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.840594240000001, 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.001 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:38:47.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:38:47.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:38:47.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:38:47.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:38:47.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:38:47.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:38:47.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:592 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 262144~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678935127999999, 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01329383, 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 592 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302562+0000", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343599999999999 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302572+0000", 2024-09-21T23:38:47.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0066e-05 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302578+0000", 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304181+0000", 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016035240000000001 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304250+0000", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8880000000000005e-05 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304345+0000", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4862000000000001e-05 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307617+0000", 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032722530000000001 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315610+0000", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0079929819999999992 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315670+0000", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9567000000000001e-05 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315722+0000", 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1958e-05 2024-09-21T23:38:47.016 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:593 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 393216~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678774901000001, 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01328609, 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 593 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302694+0000", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010562500000000001 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302702+0000", 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302706+0000", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304898+0000", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021921900000000001 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304962+0000", 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3880000000000005e-05 2024-09-21T23:38:47.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305074+0000", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011171300000000001 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307622+0000", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025482429999999999 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315786+0000", 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081640129999999995 2024-09-21T23:38:47.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315829+0000", 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3272000000000002e-05 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315874+0000", 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4733e-05 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:594 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 524288~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678649262, 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013269122, 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 594 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7e-07 2024-09-21T23:38:47.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302823+0000", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109274 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302830+0000", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1010000000000003e-06 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302835+0000", 2024-09-21T23:38:47.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305597+0000", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027625810000000001 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305660+0000", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2440999999999994e-05 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305750+0000", 2024-09-21T23:38:47.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9801000000000003e-05 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308358+0000", 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002608731 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315916+0000", 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007558085 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315953+0000", 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6588000000000003e-05 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315983+0000", 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9635999999999999e-05 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:595 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 655360~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.028 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678521143000001, 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013245257999999999, 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 595 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302946+0000", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103844 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302953+0000", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1600000000000001e-06 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302957+0000", 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306301+0000", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033432940000000001 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306364+0000", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3269999999999996e-05 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306468+0000", 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010397 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308363+0000", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018950239999999999 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316021+0000", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076579980000000001 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316059+0000", 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7957000000000003e-05 2024-09-21T23:38:47.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316087+0000", 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8038000000000002e-05 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:596 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 786432~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678398440999999, 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014017442999999999, 2024-09-21T23:38:47.034 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 596 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302965+0000", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303074+0000", 2024-09-21T23:38:47.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108726 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303082+0000", 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303087+0000", 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306999+0000", 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003911922 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307058+0000", 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9244999999999999e-05 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307152+0000", 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3790999999999997e-05 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308367+0000", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001215109 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316919+0000", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085514760000000006 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316955+0000", 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6202000000000003e-05 2024-09-21T23:38:47.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316982+0000", 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6965999999999999e-05 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:597 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 917504~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.678269163, 2024-09-21T23:38:47.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014724014000000001, 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 597 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303180+0000", 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5927000000000006e-05 2024-09-21T23:38:47.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303188+0000", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0940000000000008e-06 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303192+0000", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3320000000000002e-06 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307647+0000", 2024-09-21T23:38:47.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044549960000000001 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307726+0000", 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8580999999999999e-05 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307833+0000", 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106912 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308708+0000", 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000874565 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317661+0000", 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089531290000000006 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317753+0000", 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2410000000000005e-05 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317818+0000", 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4629999999999996e-05 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:606 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1966080~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.661449557000001, 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010800300000000001, 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.046 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 606 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319914+0000", 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320014+0000", 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100796 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320023+0000", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320029+0000", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327520+0000", 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074910100000000002 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327580+0000", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9895000000000002e-05 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327669+0000", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9441e-05 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328726+0000", 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010565749999999999 2024-09-21T23:38:47.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330656+0000", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019300999999999999 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330690+0000", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4329000000000003e-05 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330714+0000", 2024-09-21T23:38:47.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3623000000000001e-05 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:13 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.737397090999998, 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01267458, 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 13 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243968+0000", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-06 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243978+0000", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0339999999999999e-05 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244008+0000", 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9739000000000001e-05 2024-09-21T23:38:47.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244023+0000", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4912000000000001e-05 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244207+0000", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018436700000000001 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244407+0000", 2024-09-21T23:38:47.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019978199999999999 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256640+0000", 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012233173999999999 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.737302337999999, 2024-09-21T23:38:47.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0133432, 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244062+0000", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2869999999999999e-06 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244068+0000", 2024-09-21T23:38:47.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6420000000000001e-06 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244089+0000", 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0256000000000001e-05 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244104+0000", 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.518e-05 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017795899999999999 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244456+0000", 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000173697 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257404+0000", 2024-09-21T23:38:47.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012948179000000001 2024-09-21T23:38:47.062 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.062 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.062 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.737218907999999, 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.063 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:38:47.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.737089770000001, 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.067 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:47.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.737013817000001, 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014067705999999999, 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244350+0000", 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244357+0000", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2539999999999998e-06 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244381+0000", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4172000000000002e-05 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244396+0000", 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4752999999999999e-05 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244518+0000", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121882 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244724+0000", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000206665 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.258417+0000", 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013692559999999999 2024-09-21T23:38:47.075 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.705380199, 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01323613, 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.076 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275985+0000", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.469e-06 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275994+0000", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276016+0000", 2024-09-21T23:38:47.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1776e-05 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276039+0000", 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3241999999999999e-05 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276160+0000", 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120152 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276339+0000", 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017961299999999999 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289219+0000", 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012879534999999999 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:47.080 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.705285353000001, 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.705187317, 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012995227, 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276177+0000", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276182+0000", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1529999999999996e-06 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276199+0000", 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7326e-05 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276215+0000", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276330+0000", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115412 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276583+0000", 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025270700000000002 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289171+0000", 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012587783 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:27 2.3 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.596808115999998, 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012199553, 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:47.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 27 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384556+0000", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384566+0000", 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384587+0000", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1446000000000001e-05 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384598+0000", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1313e-05 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391058+0000", 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064594429999999996 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391323+0000", 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026509 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.396754+0000", 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054312609999999997 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:47.093 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:47.095 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:47.092+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:47.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:47.196 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:47.193+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:47.227 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:47.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:47.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:38:47.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:38:47.298 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:47.297+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:47.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:47.366 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:47.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:47.397 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:47.396+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:47.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:47.431 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:47.431 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd config global set global rbd_default_pool new_default_pool21555 2024-09-21T23:38:47.482 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd create --size 10M testrbdnbd21555 2024-09-21T23:38:47.498 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:47.496+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.875267183999995, 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.874559395999995, 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.872403555999995, 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.871215101999994, 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.870401442000002, 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.869929538999997, 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.825984785999999, 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.818604659000002, 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.805998023000001, 2024-09-21T23:38:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.794421697000001, 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.782950643, 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.770880637000001, 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.758886094000001, 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:47.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.747188945000001, 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:47.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.735765184000002, 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:47.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.075195975, 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.053113717, 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:47.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:47.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.027522492999999, 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:47.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:47.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:47.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.004518289, 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.990680207, 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:47.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:47.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:47.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:47.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:47.595 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:47.595 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:47.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map testrbdnbd21555 2024-09-21T23:38:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:47.597 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:47.597 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:47.598 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:38:47.598 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:47.598 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:47.600 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:47.600+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:47.607 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-21T23:38:47.232016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134808318,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3275,"num_read_kb":168198,"num_write":714,"num_write_kb":264747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":652,"ondisk_log_size":652,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":467220,"kb_used_data":267592,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565763820,"statfs":{"total":579820584960,"available":579342151680,"internally_reserved":0,"allocated":274014208,"data_stored":271239152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":653,"num_read_kb":6553,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040426"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.208682+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:38:44.208682+0000","last_peered":"2024-09-21T23:38:44.208682+0000","last_clean":"2024-09-21T23:38:44.208682+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:38:44.208682+0000","last_undegraded":"2024-09-21T23:38:44.208682+0000","last_fullsized":"2024-09-21T23:38:44.208682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"64'94","reported_seq":810,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.985469+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:38:44.985469+0000","last_peered":"2024-09-21T23:38:44.985469+0000","last_clean":"2024-09-21T23:38:44.985469+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:38:44.985469+0000","last_undegraded":"2024-09-21T23:38:44.985469+0000","last_fullsized":"2024-09-21T23:38:44.985469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:50.757146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":16777270,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":782,"num_read_kb":42259,"num_write":81,"num_write_kb":49157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"52'2","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205742+0000","last_change":"2024-09-21T23:38:24.109464+0000","last_active":"2024-09-21T23:38:44.205742+0000","last_peered":"2024-09-21T23:38:44.205742+0000","last_clean":"2024-09-21T23:38:44.205742+0000","last_became_active":"2024-09-21T23:38:22.074707+0000","last_became_peered":"2024-09-21T23:38:22.074707+0000","last_unstale":"2024-09-21T23:38:44.205742+0000","last_undegraded":"2024-09-21T23:38:44.205742+0000","last_fullsized":"2024-09-21T23:38:44.205742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:16.920782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021607010000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"60'65","reported_seq":429,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.976467+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:38:44.976467+0000","last_peered":"2024-09-21T23:38:44.976467+0000","last_clean":"2024-09-21T23:38:44.976467+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:38:44.976467+0000","last_undegraded":"2024-09-21T23:38:44.976467+0000","last_fullsized":"2024-09-21T23:38:44.976467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:31.434164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777254,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":302,"num_read_kb":26106,"num_write":64,"num_write_kb":32770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.610777+0000","last_change":"2024-09-21T23:38:24.145472+0000","last_active":"2024-09-21T23:38:44.610777+0000","last_peered":"2024-09-21T23:38:44.610777+0000","last_clean":"2024-09-21T23:38:44.610777+0000","last_became_active":"2024-09-21T23:38:22.083188+0000","last_became_peered":"2024-09-21T23:38:22.083188+0000","last_unstale":"2024-09-21T23:38:44.610777+0000","last_undegraded":"2024-09-21T23:38:44.610777+0000","last_fullsized":"2024-09-21T23:38:44.610777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:42.715435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065327500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'3","reported_seq":194,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.223655+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:38:44.223655+0000","last_peered":"2024-09-21T23:38:44.223655+0000","last_clean":"2024-09-21T23:38:44.223655+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:38:44.223655+0000","last_undegraded":"2024-09-21T23:38:44.223655+0000","last_fullsized":"2024-09-21T23:38:44.223655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:47:52.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2096,"num_write":3,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205555+0000","last_change":"2024-09-21T23:38:24.108025+0000","last_active":"2024-09-21T23:38:44.205555+0000","last_peered":"2024-09-21T23:38:44.205555+0000","last_clean":"2024-09-21T23:38:44.205555+0000","last_became_active":"2024-09-21T23:38:22.084356+0000","last_became_peered":"2024-09-21T23:38:22.084356+0000","last_unstale":"2024-09-21T23:38:44.205555+0000","last_undegraded":"2024-09-21T23:38:44.205555+0000","last_fullsized":"2024-09-21T23:38:44.205555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:07.570931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080679700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"58'41","reported_seq":282,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.905589+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:38:44.905589+0000","last_peered":"2024-09-21T23:38:44.905589+0000","last_clean":"2024-09-21T23:38:44.905589+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:38:44.905589+0000","last_undegraded":"2024-09-21T23:38:44.905589+0000","last_fullsized":"2024-09-21T23:38:44.905589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:51.834532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":20971539,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":24638,"num_write":41,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.614053+0000","last_change":"2024-09-21T23:38:24.108228+0000","last_active":"2024-09-21T23:38:44.614053+0000","last_peered":"2024-09-21T23:38:44.614053+0000","last_clean":"2024-09-21T23:38:44.614053+0000","last_became_active":"2024-09-21T23:38:22.086357+0000","last_became_peered":"2024-09-21T23:38:22.086357+0000","last_unstale":"2024-09-21T23:38:44.614053+0000","last_undegraded":"2024-09-21T23:38:44.614053+0000","last_fullsized":"2024-09-21T23:38:44.614053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:29.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498510000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.209936+0000","last_change":"2024-09-21T23:38:24.115041+0000","last_active":"2024-09-21T23:38:44.209936+0000","last_peered":"2024-09-21T23:38:44.209936+0000","last_clean":"2024-09-21T23:38:44.209936+0000","last_became_active":"2024-09-21T23:38:22.086737+0000","last_became_peered":"2024-09-21T23:38:22.086737+0000","last_unstale":"2024-09-21T23:38:44.209936+0000","last_undegraded":"2024-09-21T23:38:44.209936+0000","last_fullsized":"2024-09-21T23:38:44.209936+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:01:41.242218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057152899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"48'3","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205711+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:38:44.205711+0000","last_peered":"2024-09-21T23:38:44.205711+0000","last_clean":"2024-09-21T23:38:44.205711+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:38:44.205711+0000","last_undegraded":"2024-09-21T23:38:44.205711+0000","last_fullsized":"2024-09-21T23:38:44.205711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:37.311438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":3,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205572+0000","last_change":"2024-09-21T23:38:24.108086+0000","last_active":"2024-09-21T23:38:44.205572+0000","last_peered":"2024-09-21T23:38:44.205572+0000","last_clean":"2024-09-21T23:38:44.205572+0000","last_became_active":"2024-09-21T23:38:22.085324+0000","last_became_peered":"2024-09-21T23:38:22.085324+0000","last_unstale":"2024-09-21T23:38:44.205572+0000","last_undegraded":"2024-09-21T23:38:44.205572+0000","last_fullsized":"2024-09-21T23:38:44.205572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:27.245291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078167000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"60'124","reported_seq":452,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.208718+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:38:44.208718+0000","last_peered":"2024-09-21T23:38:44.208718+0000","last_clean":"2024-09-21T23:38:44.208718+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:38:44.208718+0000","last_undegraded":"2024-09-21T23:38:44.208718+0000","last_fullsized":"2024-09-21T23:38:44.208718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:50.973490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":20971574,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":654,"num_read_kb":18829,"num_write":188,"num_write_kb":36997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.223885+0000","last_change":"2024-09-21T23:38:24.185684+0000","last_active":"2024-09-21T23:38:44.223885+0000","last_peered":"2024-09-21T23:38:44.223885+0000","last_clean":"2024-09-21T23:38:44.223885+0000","last_became_active":"2024-09-21T23:38:22.085483+0000","last_became_peered":"2024-09-21T23:38:22.085483+0000","last_unstale":"2024-09-21T23:38:44.223885+0000","last_undegraded":"2024-09-21T23:38:44.223885+0000","last_fullsized":"2024-09-21T23:38:44.223885+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:36.524587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064978400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"61'80","reported_seq":298,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.206042+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:38:44.206042+0000","last_peered":"2024-09-21T23:38:44.206042+0000","last_clean":"2024-09-21T23:38:44.206042+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:38:44.206042+0000","last_undegraded":"2024-09-21T23:38:44.206042+0000","last_fullsized":"2024-09-21T23:38:44.206042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:04.034092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":20971558,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":218,"num_read_kb":24670,"num_write":96,"num_write_kb":45092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.610817+0000","last_change":"2024-09-21T23:38:24.145476+0000","last_active":"2024-09-21T23:38:44.610817+0000","last_peered":"2024-09-21T23:38:44.610817+0000","last_clean":"2024-09-21T23:38:44.610817+0000","last_became_active":"2024-09-21T23:38:22.084569+0000","last_became_peered":"2024-09-21T23:38:22.084569+0000","last_unstale":"2024-09-21T23:38:44.610817+0000","last_undegraded":"2024-09-21T23:38:44.610817+0000","last_fullsized":"2024-09-21T23:38:44.610817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:02.205745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060418199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'122","reported_seq":680,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.913658+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:38:44.913658+0000","last_peered":"2024-09-21T23:38:44.913658+0000","last_clean":"2024-09-21T23:38:44.913658+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:38:44.913658+0000","last_undegraded":"2024-09-21T23:38:44.913658+0000","last_fullsized":"2024-09-21T23:38:44.913658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:35:37.521533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":968,"num_read_kb":29489,"num_write":77,"num_write_kb":45061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":134217931,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3159,"num_read_kb":168098,"num_write":553,"num_write_kb":262329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268492800,"data_stored":268435862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":532,"ondisk_log_size":532,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607575,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89332,"kb_used_data":53980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282508,"statfs":{"total":96636764160,"available":96545288192,"internally_reserved":0,"allocated":55275520,"data_stored":54799820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607576,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56532,"kb_used_data":21180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315308,"statfs":{"total":96636764160,"available":96578875392,"internally_reserved":0,"allocated":21688320,"data_stored":21240280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607575,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98080,"kb_used_data":62732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94273760,"statfs":{"total":96636764160,"available":96536330240,"internally_reserved":0,"allocated":64237568,"data_stored":63773780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607575,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76960,"kb_used_data":45768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294880,"statfs":{"total":96636764160,"available":96557957120,"internally_reserved":0,"allocated":46866432,"data_stored":46406196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607574,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89264,"kb_used_data":58068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282576,"statfs":{"total":96636764160,"available":96545357824,"internally_reserved":0,"allocated":59461632,"data_stored":58994070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57052,"kb_used_data":25864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314788,"statfs":{"total":96636764160,"available":96578342912,"internally_reserved":0,"allocated":26484736,"data_stored":26025006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25174016,"data_stored":25165878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62926848,"data_stored":62914652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":54542336,"data_stored":54526063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:38:47.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:38:47.613 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:47.614 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:47.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:47.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:47.655 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:47.655 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:47.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:47.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid new_default_pool21555 2024-09-21T23:38:47.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=new_default_pool21555 2024-09-21T23:38:47.683 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:47.683 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:47.683 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''new_default_pool21555'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:47.701 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:47.700+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:47.714 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:47.714 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:47.729 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:47.747 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23518 2024-09-21T23:38:47.747 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23518 2024-09-21T23:38:47.747 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23518 -C rbd-nbd 2024-09-21T23:38:47.754 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:47.754 INFO:tasks.workunit.client.0.smithi165.stdout: 23518 ? 00:00:00 rbd-nbd 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device testrbdnbd21555 23518 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=testrbdnbd21555 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23518 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:47.755 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:47.756 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:47.756 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:47.757 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:47.757 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:47.757 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap testrbdnbd21555 2024-09-21T23:38:47.801 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:47.800+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:47.840 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:47.840 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23518\b' 2024-09-21T23:38:47.840 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23518\b' 2024-09-21T23:38:47.879 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:47.879 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:47.884 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:38:47.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:47.893 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:47.893 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:38:47.894 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23518\b' 2024-09-21T23:38:47.894 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23518\b' 2024-09-21T23:38:47.899 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:47.899 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:38:47.902 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-21T23:38:47.232016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134808318,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3275,"num_read_kb":168198,"num_write":714,"num_write_kb":264747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":652,"ondisk_log_size":652,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":467220,"kb_used_data":267592,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565763820,"statfs":{"total":579820584960,"available":579342151680,"internally_reserved":0,"allocated":274014208,"data_stored":271239152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":653,"num_read_kb":6553,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040426"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.208682+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:38:44.208682+0000","last_peered":"2024-09-21T23:38:44.208682+0000","last_clean":"2024-09-21T23:38:44.208682+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:38:44.208682+0000","last_undegraded":"2024-09-21T23:38:44.208682+0000","last_fullsized":"2024-09-21T23:38:44.208682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"64'94","reported_seq":810,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.985469+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:38:44.985469+0000","last_peered":"2024-09-21T23:38:44.985469+0000","last_clean":"2024-09-21T23:38:44.985469+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:38:44.985469+0000","last_undegraded":"2024-09-21T23:38:44.985469+0000","last_fullsized":"2024-09-21T23:38:44.985469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:50.757146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":16777270,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":782,"num_read_kb":42259,"num_write":81,"num_write_kb":49157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"52'2","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205742+0000","last_change":"2024-09-21T23:38:24.109464+0000","last_active":"2024-09-21T23:38:44.205742+0000","last_peered":"2024-09-21T23:38:44.205742+0000","last_clean":"2024-09-21T23:38:44.205742+0000","last_became_active":"2024-09-21T23:38:22.074707+0000","last_became_peered":"2024-09-21T23:38:22.074707+0000","last_unstale":"2024-09-21T23:38:44.205742+0000","last_undegraded":"2024-09-21T23:38:44.205742+0000","last_fullsized":"2024-09-21T23:38:44.205742+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:16.920782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021607010000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"60'65","reported_seq":429,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.976467+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:38:44.976467+0000","last_peered":"2024-09-21T23:38:44.976467+0000","last_clean":"2024-09-21T23:38:44.976467+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:38:44.976467+0000","last_undegraded":"2024-09-21T23:38:44.976467+0000","last_fullsized":"2024-09-21T23:38:44.976467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:03:31.434164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777254,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":302,"num_read_kb":26106,"num_write":64,"num_write_kb":32770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.610777+0000","last_change":"2024-09-21T23:38:24.145472+0000","last_active":"2024-09-21T23:38:44.610777+0000","last_peered":"2024-09-21T23:38:44.610777+0000","last_clean":"2024-09-21T23:38:44.610777+0000","last_became_active":"2024-09-21T23:38:22.083188+0000","last_became_peered":"2024-09-21T23:38:22.083188+0000","last_unstale":"2024-09-21T23:38:44.610777+0000","last_undegraded":"2024-09-21T23:38:44.610777+0000","last_fullsized":"2024-09-21T23:38:44.610777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:42.715435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065327500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'3","reported_seq":194,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.223655+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:38:44.223655+0000","last_peered":"2024-09-21T23:38:44.223655+0000","last_clean":"2024-09-21T23:38:44.223655+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:38:44.223655+0000","last_undegraded":"2024-09-21T23:38:44.223655+0000","last_fullsized":"2024-09-21T23:38:44.223655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:47:52.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2096,"num_write":3,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205555+0000","last_change":"2024-09-21T23:38:24.108025+0000","last_active":"2024-09-21T23:38:44.205555+0000","last_peered":"2024-09-21T23:38:44.205555+0000","last_clean":"2024-09-21T23:38:44.205555+0000","last_became_active":"2024-09-21T23:38:22.084356+0000","last_became_peered":"2024-09-21T23:38:22.084356+0000","last_unstale":"2024-09-21T23:38:44.205555+0000","last_undegraded":"2024-09-21T23:38:44.205555+0000","last_fullsized":"2024-09-21T23:38:44.205555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:07.570931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080679700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"58'41","reported_seq":282,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.905589+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:38:44.905589+0000","last_peered":"2024-09-21T23:38:44.905589+0000","last_clean":"2024-09-21T23:38:44.905589+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:38:44.905589+0000","last_undegraded":"2024-09-21T23:38:44.905589+0000","last_fullsized":"2024-09-21T23:38:44.905589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:51.834532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":20971539,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":24638,"num_write":41,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.614053+0000","last_change":"2024-09-21T23:38:24.108228+0000","last_active":"2024-09-21T23:38:44.614053+0000","last_peered":"2024-09-21T23:38:44.614053+0000","last_clean":"2024-09-21T23:38:44.614053+0000","last_became_active":"2024-09-21T23:38:22.086357+0000","last_became_peered":"2024-09-21T23:38:22.086357+0000","last_unstale":"2024-09-21T23:38:44.614053+0000","last_undegraded":"2024-09-21T23:38:44.614053+0000","last_fullsized":"2024-09-21T23:38:44.614053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:29.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498510000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.209936+0000","last_change":"2024-09-21T23:38:24.115041+0000","last_active":"2024-09-21T23:38:44.209936+0000","last_peered":"2024-09-21T23:38:44.209936+0000","last_clean":"2024-09-21T23:38:44.209936+0000","last_became_active":"2024-09-21T23:38:22.086737+0000","last_became_peered":"2024-09-21T23:38:22.086737+0000","last_unstale":"2024-09-21T23:38:44.209936+0000","last_undegraded":"2024-09-21T23:38:44.209936+0000","last_fullsized":"2024-09-21T23:38:44.209936+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:01:41.242218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057152899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"48'3","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205711+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:38:44.205711+0000","last_peered":"2024-09-21T23:38:44.205711+0000","last_clean":"2024-09-21T23:38:44.205711+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:38:44.205711+0000","last_undegraded":"2024-09-21T23:38:44.205711+0000","last_fullsized":"2024-09-21T23:38:44.205711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:37.311438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":3,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":43,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.205572+0000","last_change":"2024-09-21T23:38:24.108086+0000","last_active":"2024-09-21T23:38:44.205572+0000","last_peered":"2024-09-21T23:38:44.205572+0000","last_clean":"2024-09-21T23:38:44.205572+0000","last_became_active":"2024-09-21T23:38:22.085324+0000","last_became_peered":"2024-09-21T23:38:22.085324+0000","last_unstale":"2024-09-21T23:38:44.205572+0000","last_undegraded":"2024-09-21T23:38:44.205572+0000","last_fullsized":"2024-09-21T23:38:44.205572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:27.245291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078167000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"60'124","reported_seq":452,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.208718+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:38:44.208718+0000","last_peered":"2024-09-21T23:38:44.208718+0000","last_clean":"2024-09-21T23:38:44.208718+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:38:44.208718+0000","last_undegraded":"2024-09-21T23:38:44.208718+0000","last_fullsized":"2024-09-21T23:38:44.208718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:50.973490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":20971574,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":654,"num_read_kb":18829,"num_write":188,"num_write_kb":36997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.223885+0000","last_change":"2024-09-21T23:38:24.185684+0000","last_active":"2024-09-21T23:38:44.223885+0000","last_peered":"2024-09-21T23:38:44.223885+0000","last_clean":"2024-09-21T23:38:44.223885+0000","last_became_active":"2024-09-21T23:38:22.085483+0000","last_became_peered":"2024-09-21T23:38:22.085483+0000","last_unstale":"2024-09-21T23:38:44.223885+0000","last_undegraded":"2024-09-21T23:38:44.223885+0000","last_fullsized":"2024-09-21T23:38:44.223885+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:36.524587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064978400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"61'80","reported_seq":298,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.206042+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:38:44.206042+0000","last_peered":"2024-09-21T23:38:44.206042+0000","last_clean":"2024-09-21T23:38:44.206042+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:38:44.206042+0000","last_undegraded":"2024-09-21T23:38:44.206042+0000","last_fullsized":"2024-09-21T23:38:44.206042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:44:04.034092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":20971558,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":218,"num_read_kb":24670,"num_write":96,"num_write_kb":45092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.610817+0000","last_change":"2024-09-21T23:38:24.145476+0000","last_active":"2024-09-21T23:38:44.610817+0000","last_peered":"2024-09-21T23:38:44.610817+0000","last_clean":"2024-09-21T23:38:44.610817+0000","last_became_active":"2024-09-21T23:38:22.084569+0000","last_became_peered":"2024-09-21T23:38:22.084569+0000","last_unstale":"2024-09-21T23:38:44.610817+0000","last_undegraded":"2024-09-21T23:38:44.610817+0000","last_fullsized":"2024-09-21T23:38:44.610817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:02.205745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060418199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'122","reported_seq":680,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:38:44.913658+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:38:44.913658+0000","last_peered":"2024-09-21T23:38:44.913658+0000","last_clean":"2024-09-21T23:38:44.913658+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:38:44.913658+0000","last_undegraded":"2024-09-21T23:38:44.913658+0000","last_fullsized":"2024-09-21T23:38:44.913658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:35:37.521533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":968,"num_read_kb":29489,"num_write":77,"num_write_kb":45061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":134217931,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3159,"num_read_kb":168098,"num_write":553,"num_write_kb":262329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268492800,"data_stored":268435862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":532,"ondisk_log_size":532,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607575,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89332,"kb_used_data":53980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282508,"statfs":{"total":96636764160,"available":96545288192,"internally_reserved":0,"allocated":55275520,"data_stored":54799820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607576,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56532,"kb_used_data":21180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315308,"statfs":{"total":96636764160,"available":96578875392,"internally_reserved":0,"allocated":21688320,"data_stored":21240280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607575,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98080,"kb_used_data":62732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94273760,"statfs":{"total":96636764160,"available":96536330240,"internally_reserved":0,"allocated":64237568,"data_stored":63773780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607575,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76960,"kb_used_data":45768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294880,"statfs":{"total":96636764160,"available":96557957120,"internally_reserved":0,"allocated":46866432,"data_stored":46406196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607574,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89264,"kb_used_data":58068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282576,"statfs":{"total":96636764160,"available":96545357824,"internally_reserved":0,"allocated":59461632,"data_stored":58994070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57052,"kb_used_data":25864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314788,"statfs":{"total":96636764160,"available":96578342912,"internally_reserved":0,"allocated":26484736,"data_stored":26025006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25174016,"data_stored":25165878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62926848,"data_stored":62914652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":54542336,"data_stored":54526063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:38:47.903 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:38:47.903 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:47.903 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T23:38:47.904 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:38:47.904 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:38:47.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-21T23:38:47.904 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:47.900+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:47.999 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:47.999 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.003 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:48.000+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:48.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:48.056 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:48.056 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:38:48.056 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:48.113 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:48.113+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:48.135 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.135 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:48.205 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:48.204+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:48.306 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:48.308+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:48.307 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.307 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:48.401 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:38:48.401 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd config global rm global rbd_default_pool 2024-09-21T23:38:48.407 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:48.408+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:48.438 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.438 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.451 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:48.452 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:48.452 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:48.452 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:48.453 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:48.454 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:48.454 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:48.454 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:48.454 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:38:48.507 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:48.508+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:48.586 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.586 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd0 2024-09-21T23:38:48.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:48.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:48.591 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:48.592 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:48.592 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd0'\'']/id' 2024-09-21T23:38:48.601 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:48.644 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:48.644+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:48.655 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23612 2024-09-21T23:38:48.655 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23612 2024-09-21T23:38:48.655 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23612 -C rbd-nbd 2024-09-21T23:38:48.661 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:48.661 INFO:tasks.workunit.client.0.smithi165.stdout: 23612 ? 00:00:00 rbd-nbd 2024-09-21T23:38:48.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo kill 23612 2024-09-21T23:38:48.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:48.661 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:48.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:48.662 INFO:tasks.workunit.client.0.smithi165.stderr:++ which kill 2024-09-21T23:38:48.663 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/kill 2024-09-21T23:38:48.663 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:48.663 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/kill 23612 2024-09-21T23:38:48.675 INFO:tasks.workunit.client.0.smithi165.stderr:++ seq 10 2024-09-21T23:38:48.683 INFO:tasks.workunit.client.0.smithi165.stderr:+ for i in `seq 10` 2024-09-21T23:38:48.683 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:48.684 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23612 *rbd *testrbdnbd21555' 2024-09-21T23:38:48.684 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23612 *rbd *testrbdnbd21555' 2024-09-21T23:38:48.709 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:48.708+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:48.744 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:48.745 INFO:tasks.workunit.client.0.smithi165.stderr:+ break 2024-09-21T23:38:48.745 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:38:48.745 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23612 *rbd *testrbdnbd21555' 2024-09-21T23:38:48.745 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23612 *rbd *testrbdnbd21555' 2024-09-21T23:38:48.768 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.768 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.782 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:48.800 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:48.801 INFO:tasks.workunit.client.0.smithi165.stderr:+ QUIESCE_HOOK=/tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:48.801 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --quiesce --quiesce-hook /tmp/tmp.h0CUjCtN4b/quiesce.sh rbd/testrbdnbd21555 2024-09-21T23:38:48.801 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:38:48.801 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:38:48.802 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:38:48.802 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:38:48.803 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:38:48.804 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:38:48.804 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --quiesce --quiesce-hook /tmp/tmp.h0CUjCtN4b/quiesce.sh rbd/testrbdnbd21555 2024-09-21T23:38:48.828 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:48.828+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:48.911 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:48.909+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd1 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:38:48.932 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:38:48.952 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:48.953 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:48.967 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:49.012 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:49.009+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:49.018 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23674 2024-09-21T23:38:49.018 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23674 2024-09-21T23:38:49.018 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23674 -C rbd-nbd 2024-09-21T23:38:49.025 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:38:49.025 INFO:tasks.workunit.client.0.smithi165.stdout: 23674 ? 00:00:00 rbd-nbd 2024-09-21T23:38:49.026 INFO:tasks.workunit.client.0.smithi165.stderr:+ test '!' -e /tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:49.026 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd snap create rbd/testrbdnbd21555@quiesce1 2024-09-21T23:38:49.026 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@quiesce1 2024-09-21T23:38:49.094 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete...2024-09-21T23:38:49.091+0000 7fdfba860640 -1 librbd::ImageWatcher: 0x7fdf98007530 failed to notify quiesce: (1) Operation not permitted 2024-09-21T23:38:49.094 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:38:49.091+0000 7fdfba860640 -1 librbd::SnapshotCreateRequest: failed to notify quiesce: (1) Operation not permitted 2024-09-21T23:38:49.096 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.096 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.096 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete...failed. 2024-09-21T23:38:49.097 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: failed to create snapshot: (1) Operation not permitted 2024-09-21T23:38:49.112 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:49.112 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.112 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:49.112 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:49.113 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:49.109+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:49.114 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:49.114 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:49.115 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:49.115 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:49.115 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:49.115 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.150 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:38:49.150 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:38:49.150 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286365 s, 36.6 MB/s 2024-09-21T23:38:49.150 INFO:tasks.workunit.client.0.smithi165.stderr:+ touch /tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:49.152 INFO:tasks.workunit.client.0.smithi165.stderr:+ chmod +x /tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:49.152 INFO:tasks.workunit.client.0.smithi165.stderr:+ cat 2024-09-21T23:38:49.153 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@quiesce1 2024-09-21T23:38:49.213 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:49.213+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:49.239 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.239 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.253 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:49.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:38:49.315 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:49.313+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:49.377 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.377 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.391 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:49.415 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:49.413+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:49.476 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-21T23:38:49.487 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.487 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:49.487 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:49.488 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:49.489 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:49.491 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:49.492 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:49.492 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.517 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:49.513+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:49.518 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:38:49.518 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:38:49.518 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184849 s, 56.7 MB/s 2024-09-21T23:38:49.519 INFO:tasks.workunit.client.0.smithi165.stderr:++ cat /tmp/tmp.h0CUjCtN4b/quiesce 2024-09-21T23:38:49.520 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.521 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.521 INFO:tasks.workunit.client.0.smithi165.stderr:+ test /dev/nbd1 = /dev/nbd1 2024-09-21T23:38:49.521 INFO:tasks.workunit.client.0.smithi165.stderr:++ cat /tmp/tmp.h0CUjCtN4b/unquiesce 2024-09-21T23:38:49.522 INFO:tasks.workunit.client.0.smithi165.stderr:+ test /dev/nbd1 = /dev/nbd1 2024-09-21T23:38:49.523 INFO:tasks.workunit.client.0.smithi165.stderr:+ touch /tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:49.523 INFO:tasks.workunit.client.0.smithi165.stderr:+ chmod +x /tmp/tmp.h0CUjCtN4b/quiesce.sh 2024-09-21T23:38:49.524 INFO:tasks.workunit.client.0.smithi165.stderr:+ cat 2024-09-21T23:38:49.526 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false rbd snap create rbd/testrbdnbd21555@quiesce2 2024-09-21T23:38:49.526 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@quiesce2 2024-09-21T23:38:49.535 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:49.591 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 10% complete...failed. 2024-09-21T23:38:49.592 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: failed to create snapshot: (22) Invalid argument 2024-09-21T23:38:49.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:38:49.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:49.598 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:49.599 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:49.599 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:49.601 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:49.601 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:49.601 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:49.617 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:49.617+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:49.619 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:38:49.619 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:38:49.619 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114501 s, 91.6 MB/s 2024-09-21T23:38:49.620 INFO:tasks.workunit.client.0.smithi165.stderr:+ cat 2024-09-21T23:38:49.622 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@quiesce2 2024-09-21T23:38:49.657 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.657 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:49.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:49.718 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:49.717+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:49.792 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:49.792 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:49.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:49.823 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:49.824 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:49.824 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:49.821+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:49.855 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:49.856 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:49.873 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:49.874 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:49.894 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:49.894 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:49.914 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:49.920 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:49.917+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:49.932 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:49.932 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:49.933 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:49.933 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:49.934 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:49.935 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:49.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:50.022 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:50.020+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:50.110 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:50.111 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:50.123 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:50.120+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:50.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:50.223 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:50.221+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:50.236 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.608092591000002, 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.607384803000002, 2024-09-21T23:38:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.605228963000002, 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.604040509000001, 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.603226848999995, 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.602754946000005, 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.558810192999999, 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.551430066000002, 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.538823430000001, 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.527247104000001, 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.515776049999999, 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.503706044000001, 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.491711501000001, 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.480014352000001, 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.468590591000002, 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.808021382, 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.785939124, 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.760347899999999, 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.737343696, 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.723505614, 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.324 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:50.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:50.326 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:50.325+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:50.427 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:50.425+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:38:50.483 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:50.484 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:50.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:50.542 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:50.541+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:50.628 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:50.625+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:50.653 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:50.653 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:50.653 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:50.653 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:50.653 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:50.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:50.730 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:50.732+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.424330851000001, 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:38:50.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.413071509000002, 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:38:50.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:38:50.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:624 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.402275813999999, 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054719920000000002, 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 624 2024-09-21T23:38:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424346+0000", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425659+0000", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013134100000000001 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425692+0000", 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3374000000000003e-05 2024-09-21T23:38:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425712+0000", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-05 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425838+0000", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126709 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425983+0000", 2024-09-21T23:38:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014441200000000001 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.426190+0000", 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020753199999999999 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429353+0000", 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031630629999999998 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429600+0000", 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024618799999999999 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429747+0000", 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014790199999999999 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429815+0000", 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8028999999999995e-05 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.391462375, 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:38:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:38:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.345366888000001, 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:38:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.333344800999999, 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:38:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:38:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:38:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:638 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.321866314000001, 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053659859999999997, 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 638 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504755+0000", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5339999999999998e-06 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505534+0000", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077814699999999997 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505566+0000", 2024-09-21T23:38:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2487000000000003e-05 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505582+0000", 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5393e-05 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505720+0000", 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505856+0000", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013566199999999999 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.506063+0000", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206675 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509744+0000", 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003680872 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509965+0000", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221179 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510054+0000", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8899000000000004e-05 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510119+0000", 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5285999999999998e-05 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.312047861, 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:38:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:38:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.301229399, 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:38:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:38:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:684 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.048521139000002, 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055026119999999996, 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 684 2024-09-21T23:38:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778100+0000", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779170+0000", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001069576 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779203+0000", 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3170000000000003e-05 2024-09-21T23:38:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779218+0000", 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5023e-05 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779343+0000", 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012489200000000001 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779493+0000", 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014961900000000001 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779707+0000", 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021400499999999999 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783148+0000", 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034413740000000001 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783280+0000", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013202900000000001 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783491+0000", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021118499999999999 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783601+0000", 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109536 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:688 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.02802698, 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055549559999999998, 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 688 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798594+0000", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799548+0000", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095371900000000001 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799573+0000", 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-05 2024-09-21T23:38:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799587+0000", 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3499999999999999e-05 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799692+0000", 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105036 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799825+0000", 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013321200000000001 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.800022+0000", 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019666200000000001 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803718+0000", 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003696179 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803857+0000", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013872600000000001 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804047+0000", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019052500000000001 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804147+0000", 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9943000000000002e-05 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:690 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.017610270999999, 2024-09-21T23:38:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054051699999999999, 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 690 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809011+0000", 2024-09-21T23:38:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1600000000000001e-06 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809825+0000", 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081403400000000005 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809857+0000", 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2175999999999998e-05 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809873+0000", 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5207e-05 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810002+0000", 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012908900000000001 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810137+0000", 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013584900000000001 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810346+0000", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020878800000000001 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.813922+0000", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035753099999999999 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814169+0000", 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024718899999999998 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814314+0000", 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014533800000000001 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814414+0000", 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010003 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.584118837999998, 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:38:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:38:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.584026008999999, 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:38:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:38:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.247201860000001, 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:38:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:38:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:38:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:38:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.226049487000001, 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:38:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.639818655999999, 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:38:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.504489127999999, 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:128 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.299722688999999, 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054825550000000001, 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526898+0000", 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.906e-06 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527948+0000", 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010493989999999999 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527974+0000", 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6407e-05 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527988+0000", 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3720000000000001e-05 2024-09-21T23:38:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528089+0000", 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100782 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528224+0000", 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135649 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528433+0000", 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000208982 2024-09-21T23:38:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.531670+0000", 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032363919999999998 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532210+0000", 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053980599999999999 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532309+0000", 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9335999999999999e-05 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532379+0000", 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0176e-05 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.279440165, 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:50.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:38:50.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:38:50.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:50.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:50.946 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:50.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:50.950 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:50.944+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:51.047 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:51.048+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:51.068 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:51.068 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:51.068 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:51.068 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:51.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:51.148 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:51.148+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:51.187 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:51.188 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:51.188 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:51.188 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:51.188 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:51.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:51.249 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:51.249+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:51.322 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.020510431999998, 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:38:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.019646859000002, 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:38:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:38:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.018984348, 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:38:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:38:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.018310279000001, 2024-09-21T23:38:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:38:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:38:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.017583697999999, 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:38:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.016858076999998, 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:38:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.016121105, 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:38:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:652 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'27, mlcod=54'25)", 2024-09-21T23:38:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.73837279, 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063874020000000004, 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583268+0000", 2024-09-21T23:38:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584341+0000", 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072999 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584369+0000", 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7112999999999999e-05 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584382+0000", 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3672999999999999e-05 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584481+0000", 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8655999999999999e-05 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584516+0000", 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589566+0000", 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050499890000000004 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589654+0000", 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8598999999999997e-05 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:654 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'29, mlcod=54'27)", 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.724253509, 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005678677, 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597388+0000", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.72e-06 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598151+0000", 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076301400000000001 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598181+0000", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9972999999999999e-05 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598195+0000", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4059999999999999e-05 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598345+0000", 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015011199999999999 2024-09-21T23:38:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598411+0000", 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5560000000000002e-05 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.602957+0000", 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045466150000000004 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.603065+0000", 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107623 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:656 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'31, mlcod=54'29)", 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.711496754999999, 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062273229999999999, 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610145+0000", 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:38:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610983+0000", 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083891 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611010+0000", 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6927000000000002e-05 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611038+0000", 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7915999999999998e-05 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611167+0000", 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128906 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611226+0000", 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8423e-05 2024-09-21T23:38:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616286+0000", 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050602449999999997 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616370+0000", 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4486000000000004e-05 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:694 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.487172119, 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056719939999999996, 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834469+0000", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-06 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835363+0000", 2024-09-21T23:38:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089360600000000002 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835404+0000", 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0630000000000002e-05 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835424+0000", 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0123e-05 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835590+0000", 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000166625 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835656+0000", 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5510000000000001e-05 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840062+0000", 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044064239999999999 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840140+0000", 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7279000000000003e-05 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:696 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.474506359999999, 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065145940000000003, 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847135+0000", 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848041+0000", 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090629299999999996 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848068+0000", 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7453e-05 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848082+0000", 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3407000000000001e-05 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848168+0000", 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6229000000000004e-05 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848203+0000", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4480000000000002e-05 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853578+0000", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053756660000000003 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853648+0000", 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9733000000000005e-05 2024-09-21T23:38:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.375491143000001, 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:38:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:38:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.361717566999999, 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:38:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:38:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.299539163999999, 2024-09-21T23:38:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:38:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:105 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'54, mlcod=58'52)", 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.920459371, 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059755880000000004, 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401182+0000", 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402099+0000", 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091730900000000005 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402122+0000", 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402139+0000", 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6976000000000001e-05 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402233+0000", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4480000000000003e-05 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402263+0000", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9522999999999998e-05 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407054+0000", 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047915930000000002 2024-09-21T23:38:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407156+0000", 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101743 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:107 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'56, mlcod=58'54)", 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.908263979999999, 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055358780000000002, 2024-09-21T23:38:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413377+0000", 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414193+0000", 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081642699999999999 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414216+0000", 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2286999999999999e-05 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414233+0000", 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7119999999999999e-05 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414372+0000", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414405+0000", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3673000000000001e-05 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418791+0000", 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043853770000000002 2024-09-21T23:38:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418912+0000", 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121025 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:137 2.3 e58/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 58'58, mlcod=58'56)", 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.748032004000001, 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060185819999999998, 2024-09-21T23:38:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573609+0000", 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4899999999999999e-06 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574550+0000", 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094037000000000005 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574573+0000", 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3516999999999999e-05 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574585+0000", 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1712999999999999e-05 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574693+0000", 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010851199999999999 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574725+0000", 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.137e-05 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579522+0000", 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047972780000000003 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579626+0000", 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104332 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:170 2.3 e58/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 58'72, mlcod=58'70)", 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.583381946999999, 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057334100000000004, 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738260+0000", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7e-06 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739118+0000", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085804700000000002 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739147+0000", 2024-09-21T23:38:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9343e-05 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739161+0000", 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3627e-05 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739247+0000", 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6779000000000004e-05 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739280+0000", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3186999999999999e-05 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743866+0000", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045857509999999999 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743991+0000", 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124976 2024-09-21T23:38:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:175 2.3 e58/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 58'76, mlcod=58'74)", 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.557233976999999, 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061086780000000002, 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764407+0000", 2024-09-21T23:38:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.637e-06 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765253+0000", 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084597600000000002 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765283+0000", 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.915e-05 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765298+0000", 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5444e-05 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765398+0000", 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9622000000000003e-05 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765430+0000", 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2867000000000002e-05 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770401+0000", 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049709890000000003 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770514+0000", 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011299299999999999 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:51.418 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:51.420 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:51.420 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:51.417+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:51.521 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:51.520+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:51.553 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:51.553 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:51.567 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:51.655 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:51.656+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:51.723 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:51.721+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:51.732 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:51.732 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:51.732 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:51.732 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:51.732 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:51.746 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:51.835 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:51.836+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.682411127999998, 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:51.913 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:51.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:51.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.335105885000001, 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.920 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:51.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:51.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.335026452000001, 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:51.926 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:51.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.334995449000001, 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:51.930 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:51.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:51.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.334950890999998, 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:51.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:51.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.334916086, 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.939 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:51.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.311302863000002, 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:51.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:51.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.311267947000001, 2024-09-21T23:38:51.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:51.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.311236198, 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.311196537000001, 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:51.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.311165297999999, 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:51.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:51.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:51.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.310710920999998, 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:51.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:51.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:51.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.310336737, 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.981 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:51.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.308079328000002, 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.987 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:51.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:51.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:51.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:51.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.303735643, 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.993 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:51.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:51.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:51.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:51.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:51.998 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.29757202, 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:51.999 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.294342125, 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:52.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:52.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:52.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:52.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.293955869000001, 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.011 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:52.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:52.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:52.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:52.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.284194489000001, 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:52.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:52.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.283965232, 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:52.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:52.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:52.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:52.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:52.220 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:52.220 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:52.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:52.222 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:52.222 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:52.220+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:52.322 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:52.320+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:52.374 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:52.375 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:52.392 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:52.393 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:52.396 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:52.396 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:52.396 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:52.396 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:52.409 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:52.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:52.410 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:52.426 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:52.426 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:52.444 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:52.444 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:52.455 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:52.456+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:52.464 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:52.464 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:52.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:38:52.524 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:52.524+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:52.573 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:52.573 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:52.587 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:52.631 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:52.632+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:52.726 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:52.724+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:52.757 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:52.757 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:52.773 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:52.860 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:52.860+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:52.927 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:52.925+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:52.939 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:52.939 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:52.953 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:53.028 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:53.025+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:53.069 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:53.070 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:53.070 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-21T23:38:53.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-21T23:38:53.117 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.118 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.129 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:53.129+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:53.132 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:53.229 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:53.228+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:53.330 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:53.328+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:53.373 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 34 (0%) 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:avg 5.66667 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 1.97203 -> 1.97203 (expected baseline 2.17307) 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.4 with 3 -> 3 pgs (0.529412 -> 0.529412 * mean) 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.1 with 9 -> 9 pgs (1.58824 -> 1.58824 * mean) 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:oload 1000 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 3 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0008 2024-09-21T23:38:53.374 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0083 2024-09-21T23:38:53.375 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-21T23:38:53.379 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.379 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.393 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-21T23:38:53.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-21T23:38:53.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:53.431 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:53.428+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:53.480 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:38:53.545 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:53.545+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:53.552 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.552 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:53.632 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:53.632+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:53.718 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.719 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 34 (0%) 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:avg 5.66667 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 1.97203 -> 1.97203 (expected baseline 2.17307) 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.4 with 3 -> 3 pgs (0.529412 -> 0.529412 * mean) 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.1 with 9 -> 9 pgs (1.58824 -> 1.58824 * mean) 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:oload 1000 2024-09-21T23:38:53.723 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 1 2024-09-21T23:38:53.723 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-21T23:38:53.723 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0008 2024-09-21T23:38:53.723 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0083 2024-09-21T23:38:53.723 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-21T23:38:53.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:53.733 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:53.732+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:53.741 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T23:38:53.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-21T23:38:53.833 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:53.836+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:53.850 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.850 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:53.935 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:53.932+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:53.972 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:53.972 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:53.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:54.035 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:54.036+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:54.059 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:38:54.060+0000 7f95eee08640 -1 mgr.server reply reply (3) No such process no change 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 34 (0%) 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:avg 5.66667 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 1.97203 -> 1.97203 (expected baseline 2.17307) 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.4 with 3 -> 3 pgs (0.529412 -> 0.529412 * mean) 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.1 with 9 -> 9 pgs (1.58824 -> 1.58824 * mean) 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 1 2024-09-21T23:38:54.060 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0008 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0009 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 weight 1.0000 -> 0.7941 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 weight 1.0000 -> 0.8718 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 weight 1.0000 -> 0.8725 2024-09-21T23:38:54.061 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-21T23:38:54.078 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-21T23:38:54.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-21T23:38:54.094 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:54.094 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:54.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:54.135 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:54.136+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:54.236 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:54.233+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:54.272 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:54.272 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:54.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 34 (0%) 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout:avg 5.66667 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 1.97203 -> 1.97203 (expected baseline 2.17307) 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.4 with 3 -> 3 pgs (0.529412 -> 0.529412 * mean) 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.1 with 9 -> 9 pgs (1.58824 -> 1.58824 * mean) 2024-09-21T23:38:54.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stdout:oload 1000 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 1 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0008 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0083 2024-09-21T23:38:54.371 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-21T23:38:54.371 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:54.369+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:54.388 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T23:38:54.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-21T23:38:54.435 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:54.436 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:54.436 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:54.437+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:54.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:54.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:54.538 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:54.536+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:54.608 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:54.608 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:54.622 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:54.629 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:54.645 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:54.645 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:54.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:54.662 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:54.667 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:54.668+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:54.680 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:54.680 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:54.698 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:54.698 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:54.717 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:54.717 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:54.740 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:54.740+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:54.741 INFO:tasks.ceph.mgr.x.smithi118.stderr:2024-09-21T23:38:54.740+0000 7f95eee08640 -1 mgr.server reply reply (3) No such process no change 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 34 (0%) 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout:avg 5.66667 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 1.97203 -> 1.97203 (expected baseline 2.17307) 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.4 with 3 -> 3 pgs (0.529412 -> 0.529412 * mean) 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.1 with 9 -> 9 pgs (1.58824 -> 1.58824 * mean) 2024-09-21T23:38:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 0.05 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0008 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0009 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-21T23:38:54.743 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-21T23:38:54.756 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:54.756 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:54.756 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:54.756 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:54.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:54.840 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:54.837+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:54.929 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:54.929 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:54.929 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:54.929 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:54.929 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:54.941 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:54.940+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:54.944 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:55.042 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:55.040+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:55.115 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:55.115 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:55.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:55.115 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.88452313099999, 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.116 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:38:55.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:38:55.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.537217888000001, 2024-09-21T23:38:55.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:38:55.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:38:55.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.537138455000001, 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:55.128 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:38:55.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.537107452000001, 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.132 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:38:55.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.537062894000002, 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.136 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:38:55.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.537028089, 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:38:55.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:55.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.513414866000002, 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.144 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:38:55.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.513379950000001, 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:38:55.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:38:55.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.155 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.513348200999999, 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:38:55.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.513308540000001, 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:38:55.161 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:38:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:38:55.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:38:55.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.513277300999999, 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.167 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:38:55.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.172 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.512822923999998, 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:38:55.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:38:55.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:38:55.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.51244874, 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:38:55.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:38:55.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.184 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.510191331000001, 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:55.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.505847645999999, 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.195 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.499684023, 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.196 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:38:55.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:38:55.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:38:55.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.496454128, 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.202 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:38:55.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:55.207 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.496067872000001, 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:38:55.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:38:55.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.486306492000001, 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.213 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:38:55.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:38:55.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.486077235, 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.224 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.225 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.225 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:55.226 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:55.227 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:55.225+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:55.327 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:55.325+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:55.393 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:55.394 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:55.407 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:55.428 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:55.429+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:55.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:38:55.526 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:55.526 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:55.526 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:55.527 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:55.527 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:55.540 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:55.545 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:55.545+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:55.630 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:55.629+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:55.661 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:55.661 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:55.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:55.661 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.298620903, 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:38:55.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.286956864, 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.666 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:38:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.275390635000001, 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.789453098999999, 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.675 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.679 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:702 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'32, mlcod=55'30)", 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.781424136999998, 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002866164, 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880719+0000", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881020+0000", 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000301341 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881038+0000", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7932e-05 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881049+0000", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0523000000000001e-05 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881122+0000", 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3595000000000003e-05 2024-09-21T23:38:55.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881153+0000", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0102000000000001e-05 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883546+0000", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023938890000000002 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883584+0000", 2024-09-21T23:38:55.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7540999999999999e-05 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:704 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'34, mlcod=55'32)", 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.773387949, 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027588729999999998, 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.684 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888755+0000", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.093e-06 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889090+0000", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033504800000000001 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889109+0000", 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8627999999999999e-05 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889119+0000", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7219999999999994e-06 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889189+0000", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0308999999999994e-05 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889221+0000", 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1755999999999999e-05 2024-09-21T23:38:55.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891424+0000", 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022030650000000001 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891513+0000", 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9252000000000005e-05 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.688 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.765542981999999, 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:38:55.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:38:55.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.757051920999999, 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:38:55.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:38:55.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:710 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'40, mlcod=55'38)", 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.746512009, 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002832621, 2024-09-21T23:38:55.697 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915631+0000", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.158e-06 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916242+0000", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061049200000000002 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916260+0000", 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7938e-05 2024-09-21T23:38:55.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916268+0000", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5590000000000006e-06 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916328+0000", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0189999999999998e-05 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916350+0000", 2024-09-21T23:38:55.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2201e-05 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918425+0000", 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074944 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918463+0000", 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7138999999999998e-05 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.737916337000001, 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:38:55.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.728512017, 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.710 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.083076891000001, 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:38:55.714 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.051422508000002, 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:38:55.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.047719434000001, 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.719 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:38:55.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.492116005, 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:38:55.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:38:55.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.033568754000001, 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.728 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:38:55.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.022205496000002, 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:38:55.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.010810539000001, 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.741 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.0012018, 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.745 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5487.0:6 3.3 3:daeef835:::rbd_id.testrbdnbd21555:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.1327210109999992, 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027352959999999999, 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5487", 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2667436824", 2024-09-21T23:38:55.746 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 6 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529423+0000", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7400000000000001e-06 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529434+0000", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1643e-05 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529457+0000", 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2996000000000001e-05 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529479+0000", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1953000000000001e-05 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529629+0000", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000149729 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529878+0000", 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024851400000000003 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.530224+0000", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034664199999999998 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.531634+0000", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001410288 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.531877+0000", 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024207100000000001 2024-09-21T23:38:55.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.532034+0000", 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015757299999999999 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.532156+0000", 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012214699999999999 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:55.751 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:55.753 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:55.753 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:55.756+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:55.853 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:55.852+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:55.921 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:55.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:55.922 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:38:55.922 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:55.935 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:55.979 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:55.980+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:56.055 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:56.053+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:56.104 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:56.104 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:38:56.104 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:38:56.104 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:38:56.104 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:56.118 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:56.163 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:56.164+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:56.243 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.148277426999996, 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:56.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:38:56.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.134199879999997, 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:56.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:38:56.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:38:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.125108068999999, 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:38:56.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:38:56.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.103060972999998, 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:38:56.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:592 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 262144~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.941401860999999, 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01329383, 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.268 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 592 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302562+0000", 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343599999999999 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302572+0000", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0066e-05 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302578+0000", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304181+0000", 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016035240000000001 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304250+0000", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8880000000000005e-05 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304345+0000", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4862000000000001e-05 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307617+0000", 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032722530000000001 2024-09-21T23:38:56.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315610+0000", 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0079929819999999992 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315670+0000", 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9567000000000001e-05 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315722+0000", 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1958e-05 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:593 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 393216~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.941241634000001, 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01328609, 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 593 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:56.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302694+0000", 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010562500000000001 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302702+0000", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302706+0000", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304898+0000", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021921900000000001 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304962+0000", 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3880000000000005e-05 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305074+0000", 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011171300000000001 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307622+0000", 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025482429999999999 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315786+0000", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081640129999999995 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315829+0000", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3272000000000002e-05 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315874+0000", 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4733e-05 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:594 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 524288~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.941115995000001, 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013269122, 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.280 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 594 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7e-07 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302823+0000", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109274 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302830+0000", 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1010000000000003e-06 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302835+0000", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305597+0000", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027625810000000001 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305660+0000", 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2440999999999994e-05 2024-09-21T23:38:56.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305750+0000", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9801000000000003e-05 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308358+0000", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002608731 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315916+0000", 2024-09-21T23:38:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007558085 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315953+0000", 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6588000000000003e-05 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315983+0000", 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9635999999999999e-05 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:595 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 655360~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.940987876000001, 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013245257999999999, 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 595 2024-09-21T23:38:56.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:56.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302946+0000", 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103844 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302953+0000", 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1600000000000001e-06 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302957+0000", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306301+0000", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033432940000000001 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306364+0000", 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3269999999999996e-05 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306468+0000", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010397 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308363+0000", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018950239999999999 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316021+0000", 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076579980000000001 2024-09-21T23:38:56.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316059+0000", 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7957000000000003e-05 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316087+0000", 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8038000000000002e-05 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:596 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 786432~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.940865173999999, 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014017442999999999, 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 596 2024-09-21T23:38:56.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302965+0000", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303074+0000", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108726 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303082+0000", 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303087+0000", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306999+0000", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003911922 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307058+0000", 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9244999999999999e-05 2024-09-21T23:38:56.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307152+0000", 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3790999999999997e-05 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308367+0000", 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001215109 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316919+0000", 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085514760000000006 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316955+0000", 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6202000000000003e-05 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316982+0000", 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6965999999999999e-05 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.297 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:597 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 917504~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.940735896, 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014724014000000001, 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 597 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:56.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303180+0000", 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5927000000000006e-05 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303188+0000", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0940000000000008e-06 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303192+0000", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3320000000000002e-06 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307647+0000", 2024-09-21T23:38:56.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044549960000000001 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307726+0000", 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8580999999999999e-05 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307833+0000", 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106912 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308708+0000", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000874565 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317661+0000", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089531290000000006 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317753+0000", 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2410000000000005e-05 2024-09-21T23:38:56.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317818+0000", 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4629999999999996e-05 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:606 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1966080~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.923916290000001, 2024-09-21T23:38:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010800300000000001, 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 606 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319914+0000", 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320014+0000", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100796 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320023+0000", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320029+0000", 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:38:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327520+0000", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074910100000000002 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327580+0000", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9895000000000002e-05 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327669+0000", 2024-09-21T23:38:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9441e-05 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328726+0000", 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010565749999999999 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330656+0000", 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019300999999999999 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330690+0000", 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4329000000000003e-05 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330714+0000", 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3623000000000001e-05 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:13 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.999863823999998, 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01267458, 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 13 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243968+0000", 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-06 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243978+0000", 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0339999999999999e-05 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244008+0000", 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9739000000000001e-05 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244023+0000", 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4912000000000001e-05 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244207+0000", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018436700000000001 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244407+0000", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019978199999999999 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256640+0000", 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012233173999999999 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.999769070999999, 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0133432, 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244062+0000", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2869999999999999e-06 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244068+0000", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6420000000000001e-06 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244089+0000", 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0256000000000001e-05 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244104+0000", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.518e-05 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017795899999999999 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244456+0000", 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000173697 2024-09-21T23:38:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257404+0000", 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012948179000000001 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.999685640999999, 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:38:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:38:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.999556503000001, 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:38:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.999480550000001, 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014067705999999999, 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244350+0000", 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244357+0000", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2539999999999998e-06 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244381+0000", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4172000000000002e-05 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244396+0000", 2024-09-21T23:38:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4752999999999999e-05 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244518+0000", 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121882 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244724+0000", 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000206665 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.258417+0000", 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013692559999999999 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.967846932, 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01323613, 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275985+0000", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.469e-06 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275994+0000", 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276016+0000", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1776e-05 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276039+0000", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3241999999999999e-05 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276160+0000", 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120152 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276339+0000", 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017961299999999999 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289219+0000", 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012879534999999999 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.967752086000001, 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:38:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:38:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.96765405, 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012995227, 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276177+0000", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276182+0000", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1529999999999996e-06 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276199+0000", 2024-09-21T23:38:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7326e-05 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276215+0000", 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276330+0000", 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115412 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276583+0000", 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025270700000000002 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289171+0000", 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012587783 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:27 2.3 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.859274848999998, 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012199553, 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 27 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384556+0000", 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384566+0000", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384587+0000", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1446000000000001e-05 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384598+0000", 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1313e-05 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391058+0000", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064594429999999996 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391323+0000", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026509 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.396754+0000", 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054312609999999997 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:56.351 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:56.352 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:56.352+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:56.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:56.453 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:56.456+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:56.486 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:56.486 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:56.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:56.498 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:38:56.561 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:56.561+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:56.616 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:56.616 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:56.617 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:56.617 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:56.617 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:56.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:56.654 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:38:56.656+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:38:56.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:38:56.754 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:56.752+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.169000522, 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.168292734, 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.166136894, 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.16494844, 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.16413478, 2024-09-21T23:38:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.16366287699999, 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.119718123999998, 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.112337997000001, 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.099731361, 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.088155035, 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.076683980999999, 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.064613975, 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.052619432, 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.040922283, 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.029498522000001, 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.368929312999999, 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.346847055000001, 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.321255830999998, 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.298251626999999, 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:56.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.284413545, 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:56.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:56.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:56.889 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:56.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:56.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:56.891 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:56.892 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:56.889+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:56.892 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:56.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:56.908 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:56.908 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:56.928 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:56.928 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:56.949 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:56.949 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:56.969 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:56.970 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:56.991 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:56.988+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:57.008 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.008 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:38:57.109 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:57.109+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:57.181 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.181 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:38:57.195 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:57.196+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:57.294 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:57.293+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:57.363 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.363 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:38:57.421 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:57.421+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:57.496 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:57.493+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:57.522 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 20% complete... Creating snap: 100% complete...done. 2024-09-21T23:38:57.532 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:57.532 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:57.533 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:57.534 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:57.534 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:38:57.536 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:38:57.536 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:57.537 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/dev/nbd1 bs=1M count=1 oflag=direct 2024-09-21T23:38:57.546 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.547 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:38:57.565 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:38:57.565 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:38:57.565 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209874 s, 50.0 MB/s 2024-09-21T23:38:57.566 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd1 23674 2024-09-21T23:38:57.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd1 2024-09-21T23:38:57.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23674 2024-09-21T23:38:57.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:38:57.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:38:57.567 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:38:57.568 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:38:57.569 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:38:57.570 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:38:57.570 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:38:57.570 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:38:57.597 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:57.596+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:57.698 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:57.700+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:57.728 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.729 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:38:57.799 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:57.797+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:57.860 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:57.860 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:57.875 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:38:57.920 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:57.920+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:58.001 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:58.001+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:58.008 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.009 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.023 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:38:58.111 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:58.112+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:58.193 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.193 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.202 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:58.201+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:58.208 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:38:58.303 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:58.301+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:58.373 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.373 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.388 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:38:58.435 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:58.436+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:58.505 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:58.505+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:58.513 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.513 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:38:58.528 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:38:58.620 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:58.620+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:58.698 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.699 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.707 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:58.705+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:58.712 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:38:58.808 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:58.805+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:58.880 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:58.880 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:58.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:38:58.908 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:38:58.905+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:38:58.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:38:59.016 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:59.017 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:59.031 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:38:59.037 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:59.037+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:59.110 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:59.109+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:59.135 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:38:59.135 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:38:59.151 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:38:59.152 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:38:59.154 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:38:59.154 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:38:59.168 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:38:59.168 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:38:59.169 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:38:59.187 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:38:59.187 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:38:59.205 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:38:59.205 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:38:59.215 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:38:59.216+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:38:59.224 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:38:59.224 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:38:59.294 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:38:59.294 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:38:59.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:38:59.294 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:38:59.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 39.199099572000002, 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.295 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:38:59.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:38:59.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:59.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 39.185022025000002, 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:59.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:38:59.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 39.175930213999997, 2024-09-21T23:38:59.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:59.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:38:59.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:38:59.312 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 39.153883118000003, 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:38:59.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:592 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 262144~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.318 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.992224006000001, 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01329383, 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 592 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302428+0000", 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1999999999999999e-07 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302562+0000", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013343599999999999 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302572+0000", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0066e-05 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302578+0000", 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5820000000000001e-06 2024-09-21T23:38:59.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304181+0000", 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016035240000000001 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304250+0000", 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8880000000000005e-05 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304345+0000", 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4862000000000001e-05 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307617+0000", 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032722530000000001 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315610+0000", 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0079929819999999992 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315670+0000", 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9567000000000001e-05 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315722+0000", 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1958e-05 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:593 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 393216~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.992063778999999, 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01328609, 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 593 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:59.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302588+0000", 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302694+0000", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010562500000000001 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302702+0000", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302706+0000", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304898+0000", 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021921900000000001 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.304962+0000", 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3880000000000005e-05 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305074+0000", 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011171300000000001 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307622+0000", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025482429999999999 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315786+0000", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081640129999999995 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315829+0000", 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3272000000000002e-05 2024-09-21T23:38:59.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315874+0000", 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4733e-05 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:594 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 524288~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.991938139999998, 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013269122, 2024-09-21T23:38:59.330 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 594 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302714+0000", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7e-07 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302823+0000", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109274 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302830+0000", 2024-09-21T23:38:59.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1010000000000003e-06 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302835+0000", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305597+0000", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027625810000000001 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305660+0000", 2024-09-21T23:38:59.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2440999999999994e-05 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.305750+0000", 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9801000000000003e-05 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308358+0000", 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002608731 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315916+0000", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007558085 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315953+0000", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6588000000000003e-05 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.315983+0000", 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9635999999999999e-05 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.335 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:595 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 655360~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.991810020999999, 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013245257999999999, 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 595 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.336 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302842+0000", 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302946+0000", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103844 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302953+0000", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1600000000000001e-06 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302957+0000", 2024-09-21T23:38:59.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306301+0000", 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033432940000000001 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306364+0000", 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3269999999999996e-05 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306468+0000", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010397 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308363+0000", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018950239999999999 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316021+0000", 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076579980000000001 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316059+0000", 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7957000000000003e-05 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316087+0000", 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8038000000000002e-05 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.341 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:596 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 786432~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.991687319, 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014017442999999999, 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 596 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302964+0000", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.302965+0000", 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:38:59.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303074+0000", 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108726 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303082+0000", 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303087+0000", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.306999+0000", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003911922 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307058+0000", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9244999999999999e-05 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307152+0000", 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3790999999999997e-05 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308367+0000", 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001215109 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316919+0000", 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085514760000000006 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316955+0000", 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6202000000000003e-05 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.316982+0000", 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6965999999999999e-05 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.347 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:597 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 917504~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.991558041000001, 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014724014000000001, 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 597 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303094+0000", 2024-09-21T23:38:59.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303180+0000", 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5927000000000006e-05 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303188+0000", 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0940000000000008e-06 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.303192+0000", 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3320000000000002e-06 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307647+0000", 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044549960000000001 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307726+0000", 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8580999999999999e-05 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.307833+0000", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106912 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.308708+0000", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000874565 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317661+0000", 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089531290000000006 2024-09-21T23:38:59.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317753+0000", 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2410000000000005e-05 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.317818+0000", 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4629999999999996e-05 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4754.0:606 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 1966080~131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.974738434999999, 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010800300000000001, 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4754", 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 606 2024-09-21T23:38:59.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319913+0000", 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.319914+0000", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2e-07 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320014+0000", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100796 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320023+0000", 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.320029+0000", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327520+0000", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074910100000000002 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327580+0000", 2024-09-21T23:38:59.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9895000000000002e-05 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.327669+0000", 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9441e-05 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.328726+0000", 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010565749999999999 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:38:59.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330656+0000", 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019300999999999999 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330690+0000", 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4329000000000003e-05 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.330714+0000", 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3623000000000001e-05 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:13 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.050685969, 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01267458, 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 13 2024-09-21T23:38:59.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243966+0000", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243968+0000", 2024-09-21T23:38:59.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-06 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.243978+0000", 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0339999999999999e-05 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244008+0000", 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9739000000000001e-05 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244023+0000", 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4912000000000001e-05 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244207+0000", 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018436700000000001 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244407+0000", 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019978199999999999 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256640+0000", 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012233173999999999 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.050591216000001, 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0133432, 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.364 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244061+0000", 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244062+0000", 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2869999999999999e-06 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244068+0000", 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6420000000000001e-06 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244089+0000", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0256000000000001e-05 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244104+0000", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.518e-05 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017795899999999999 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244456+0000", 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000173697 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257404+0000", 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012948179000000001 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.368 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.050507786000001, 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:38:59.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.050378647999999, 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.373 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.050302694999999, 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014067705999999999, 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244349+0000", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244350+0000", 2024-09-21T23:38:59.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.42e-06 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244357+0000", 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2539999999999998e-06 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244381+0000", 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4172000000000002e-05 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244396+0000", 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4752999999999999e-05 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244518+0000", 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121882 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244724+0000", 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000206665 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.258417+0000", 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013692559999999999 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:14 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.018669076999998, 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01323613, 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.382 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 14 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275983+0000", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275985+0000", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.469e-06 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.275994+0000", 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276016+0000", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1776e-05 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276039+0000", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3241999999999999e-05 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276160+0000", 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120152 2024-09-21T23:38:59.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276339+0000", 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017961299999999999 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289219+0000", 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012879534999999999 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.018574230999999, 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.387 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:38:59.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:20 2.3 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.018476195000002, 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012995227, 2024-09-21T23:38:59.391 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276176+0000", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276177+0000", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276182+0000", 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1529999999999996e-06 2024-09-21T23:38:59.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276199+0000", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7326e-05 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276215+0000", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5835999999999999e-05 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276330+0000", 2024-09-21T23:38:59.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115412 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276583+0000", 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025270700000000002 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289171+0000", 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012587783 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:27 2.3 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 30.910096994, 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012199553, 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 27 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:38:59.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384555+0000", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384556+0000", 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:38:59.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384566+0000", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7100000000000002e-06 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384587+0000", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1446000000000001e-05 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.384598+0000", 2024-09-21T23:38:59.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1313e-05 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391058+0000", 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064594429999999996 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.391323+0000", 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026509 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:38:59.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.396754+0000", 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054312609999999997 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:38:59.400 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:38:59.401 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:59.400+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:59.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:38:59.502 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:59.501+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:38:59.511 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:38:59.532 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:38:59.532 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:59.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:38:59.603 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:59.604+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:59.704 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:38:59.701+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:38:59.715 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:59.715 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:38:59.715 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:38:59.715 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:38:59.715 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:59.729 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:38:59.761 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:38:59.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:38:59.761 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T23:38:59.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:38:59.805 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:38:59.804+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.26313564100001, 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:38:59.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:38:59.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:132 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'90, mlcod=16'85)", 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.26242785300001, 2024-09-21T23:38:59.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042575499999999997, 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628970+0000", 2024-09-21T23:38:59.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628971+0000", 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628984+0000", 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2687000000000001e-05 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629055+0000", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0696000000000002e-05 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629066+0000", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629712+0000", 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064654899999999995 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.629745+0000", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3043000000000002e-05 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633192+0000", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034469399999999999 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633228+0000", 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5478999999999997e-05 2024-09-21T23:38:59.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:135 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.26027201300001, 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045101380000000003, 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631126+0000", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631127+0000", 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4130000000000001e-06 2024-09-21T23:38:59.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631142+0000", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4924e-05 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631163+0000", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1115999999999999e-05 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631179+0000", 2024-09-21T23:38:59.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5406000000000001e-05 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631928+0000", 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074935799999999999 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.631960+0000", 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1979e-05 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635566+0000", 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036055089999999998 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635636+0000", 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0432999999999995e-05 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:38:59.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.259083559, 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:38:59.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.258269899, 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:38:59.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:38:59.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:139 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'97, mlcod=16'87)", 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 106.25779799599999, 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042179339999999996, 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633600+0000", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633601+0000", 2024-09-21T23:38:59.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633613+0000", 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2156e-05 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633640+0000", 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7042999999999998e-05 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633656+0000", 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-05 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634994+0000", 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013383340000000001 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.635041+0000", 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829000000000002e-05 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637782+0000", 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027405950000000002 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637818+0000", 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5859999999999999e-05 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:38:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.213853243000003, 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:38:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:38:59.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.206473115999998, 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:38:59.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:38:59.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:670 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'30, mlcod=54'28)", 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.193866479999997, 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045937419999999996, 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697531+0000", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.697533+0000", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698571+0000", 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010377559999999999 2024-09-21T23:38:59.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698596+0000", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5593e-05 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698609+0000", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3040000000000001e-05 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698764+0000", 2024-09-21T23:38:59.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424199999999999 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.698800+0000", 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6152999999999998e-05 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702031+0000", 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032311620000000001 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.702125+0000", 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4213000000000001e-05 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:672 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'32, mlcod=54'30)", 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.182290154, 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044987059999999999, 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709108+0000", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.709109+0000", 2024-09-21T23:38:59.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.714e-06 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710100+0000", 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0009901790000000001 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710126+0000", 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6432999999999999e-05 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710140+0000", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4012999999999999e-05 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710317+0000", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017735900000000001 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.710354+0000", 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6207000000000001e-05 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713478+0000", 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031243220000000001 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.713606+0000", 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128479 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.937 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:674 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'34, mlcod=54'32)", 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.170819100000003, 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004485396, 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720579+0000", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.720581+0000", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721515+0000", 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093444999999999997 2024-09-21T23:38:59.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721544+0000", 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8935999999999999e-05 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721558+0000", 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3502999999999999e-05 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721739+0000", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000180979 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.721776+0000", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7570000000000001e-05 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.724971+0000", 2024-09-21T23:38:59.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031945530000000001 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.725064+0000", 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3362000000000005e-05 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:59.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.158749094000001, 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:38:59.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:38:59.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.146754551000001, 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.947 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:38:59.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:680 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'40, mlcod=54'38)", 2024-09-21T23:38:59.951 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.135057402000001, 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044552200000000002, 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756340+0000", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.756343+0000", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1270000000000001e-06 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757276+0000", 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093378899999999997 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757306+0000", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9164000000000001e-05 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757319+0000", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3593e-05 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757429+0000", 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010993900000000001 2024-09-21T23:38:59.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.757462+0000", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2682999999999999e-05 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760720+0000", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032584490000000001 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.760796+0000", 2024-09-21T23:38:59.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5475999999999994e-05 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:682 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'42, mlcod=54'40)", 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.123633640999998, 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044921300000000004, 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767764+0000", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.767766+0000", 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768697+0000", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093066299999999998 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768726+0000", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9335999999999998e-05 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768740+0000", 2024-09-21T23:38:59.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3796e-05 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768862+0000", 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012142900000000001 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.768899+0000", 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7806999999999999e-05 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772178+0000", 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032785179999999998 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.772256+0000", 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8397000000000002e-05 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:59.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.463064431999999, 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:38:59.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.964 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:114 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'101, mlcod=58'99)", 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.440982173999998, 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042029679999999996, 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450416+0000", 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.450417+0000", 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451310+0000", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089362700000000003 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451331+0000", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1069000000000001e-05 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451347+0000", 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5362999999999998e-05 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451445+0000", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8376999999999997e-05 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.451479+0000", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4316000000000001e-05 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454526+0000", 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003046207 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.454619+0000", 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.415390949999999, 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:38:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.392386746, 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:38:59.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:38:59.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.378548664, 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:38:59.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:38:59.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:38:59.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:38:59.983 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:38:59.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:38:59.985 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:38:59.981+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:00.074 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:00.074 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 73, 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66973394-6d88-4f14-bf11-7ccd7c9cf4a7", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-21T23:36:58.086363+0000", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-21T23:38:59.502713+0000", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-21T23:37:09.181667+0000", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-21T23:36:59.825771+0000", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-21T23:39:00.075 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 6, 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "squid", 2024-09-21T23:39:00.076 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:11.018666+0000", 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:39:00.077 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:39:00.078 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "16", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-21T23:39:00.079 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:39:00.080 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:39:00.081 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:39:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.97197729349136353, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:15.320283+0000", 2024-09-21T23:39:00.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-21T23:39:00.084 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "72", 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:39:00.085 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 6, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 72, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:39:00.086 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:39:00.087 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:39:00.088 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.25, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.25, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.25, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.25, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.97197729349136353, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "new_default_pool21555", 2024-09-21T23:39:00.089 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:38:20.841333+0000", 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:39:00.090 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "52", 2024-09-21T23:39:00.091 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 2, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 52, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:39:00.092 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:39:00.093 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-21T23:39:00.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.2699999809265137, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.2699999809265137, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.82999998331069946, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.97197729349136353, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:39:00.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "638e3e48-1763-4ec9-8c06-634e0b3b793b", 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.096 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/915397992", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/915397992", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/915397992", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/915397992", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "02967526-d5c5-4f68-8e25-343311dbaec4", 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.961090087890625, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-21T23:39:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-21T23:39:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-21T23:39:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/3900686792", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/3900686792", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/3900686792", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/3900686792", 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "4f770ac1-d062-4934-81d5-7f980b3830a1", 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.8707733154296875, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-21T23:39:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/2564679849", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/2564679849", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6823/2564679849", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/2564679849", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "90cc3313-3845-48d0-839d-8fea4c52bfde", 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 17, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6808", 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6809", 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6810", 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6811", 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6814", 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6815", 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6812", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6813", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6809/1410364236", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6811/1410364236", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6815/1410364236", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6813/1410364236", 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "42b8e641-af88-4d41-bc5f-a14020f205ce", 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6800", 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6801", 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6802", 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6803", 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6806", 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6807", 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6804", 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6805", 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6801/1572346429", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6803/1572346429", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6807/1572346429", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6805/1572346429", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9", 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:00.123 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6816", 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6817", 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6818", 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6819", 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6822", 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6823", 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6820", 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:00.127 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6821", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6817/466572374", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6819/466572374", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6823/466572374", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6821/466572374", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:00.128 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.810277+0000", 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.129 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.686068+0000", 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.130 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.949768+0000", 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.773303+0000", 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.131 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.009610+0000", 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:00.132 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.059224+0000", 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.133 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-21T23:39:00.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.135 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.137 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': '66973394-6d88-4f14-bf11-7ccd7c9cf4a7', 'created': '2024-09-21T23:36:58.086363+0000', 'modified': '2024-09-21T23:38:59.502713+0000', 'last_up_change': '2024-09-21T23:37:09.181667+0000', 'last_in_change': '2024-09-21T23:36:59.825771+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:37:11.018666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9719772934913635, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:37:15.320283+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 72, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9719772934913635, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'new_default_pool21555', 'create_time': '2024-09-21T23:38:20.841333+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9719772934913635, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '638e3e48-1763-4ec9-8c06-634e0b3b793b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 915397992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 915397992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 915397992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 915397992}]}, 'public_addr': '172.21.15.79:6801/915397992', 'cluster_addr': '172.21.15.79:6803/915397992', 'heartbeat_back_addr': '172.21.15.79:6807/915397992', 'heartbeat_front_addr': '172.21.15.79:6805/915397992', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02967526-d5c5-4f68-8e25-343311dbaec4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.961090087890625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 3900686792}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 3900686792}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 3900686792}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 3900686792}]}, 'public_addr': '172.21.15.79:6809/3900686792', 'cluster_addr': '172.21.15.79:6811/3900686792', 'heartbeat_back_addr': '172.21.15.79:6815/3900686792', 'heartbeat_front_addr': '172.21.15.79:6813/3900686792', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4f770ac1-d062-4934-81d5-7f980b3830a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8707733154296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2564679849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 2564679849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 2564679849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 2564679849}]}, 'public_addr': '172.21.15.79:6817/2564679849', 'cluster_addr': '172.21.15.79:6819/2564679849', 'heartbeat_back_addr': '172.21.15.79:6823/2564679849', 'heartbeat_front_addr': '172.21.15.79:6821/2564679849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '90cc3313-3845-48d0-839d-8fea4c52bfde', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 1410364236}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 1410364236}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 1410364236}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 1410364236}]}, 'public_addr': '172.21.15.118:6809/1410364236', 'cluster_addr': '172.21.15.118:6811/1410364236', 'heartbeat_back_addr': '172.21.15.118:6815/1410364236', 'heartbeat_front_addr': '172.21.15.118:6813/1410364236', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '42b8e641-af88-4d41-bc5f-a14020f205ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 1572346429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 1572346429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 1572346429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 1572346429}]}, 'public_addr': '172.21.15.118:6801/1572346429', 'cluster_addr': '172.21.15.118:6803/1572346429', 'heartbeat_back_addr': '172.21.15.118:6807/1572346429', 'heartbeat_front_addr': '172.21.15.118:6805/1572346429', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6817', 'nonce': 466572374}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6818', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 466572374}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6822', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 466572374}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6821', 'nonce': 466572374}]}, 'public_addr': '172.21.15.118:6817/466572374', 'cluster_addr': '172.21.15.118:6819/466572374', 'heartbeat_back_addr': '172.21.15.118:6823/466572374', 'heartbeat_front_addr': '172.21.15.118:6821/466572374', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.810277+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.686068+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.949768+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.773303+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.009610+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.059224+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.0', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:39:00.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:00.138 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:00.137+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:00.155 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.155 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:00.155 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:00.156 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:00.257 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:00.257+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:00.325 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.325 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:00.325 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:00.326 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:00.326 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:00.339 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:00.337+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:00.439 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:00.437+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:00.460 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:00.478 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-21T23:38:59.512748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136905559,"num_objects":57,"num_object_clones":2,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":3874,"num_read_kb":171760,"num_write":750,"num_write_kb":267831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":710,"ondisk_log_size":710,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":471784,"kb_used_data":272168,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565759256,"statfs":{"total":579820584960,"available":579337478144,"internally_reserved":0,"allocated":278700032,"data_stored":288207610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.034311"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":270,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.482480+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:38:53.482480+0000","last_peered":"2024-09-21T23:38:53.482480+0000","last_clean":"2024-09-21T23:38:53.482480+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:38:53.482480+0000","last_undegraded":"2024-09-21T23:38:53.482480+0000","last_fullsized":"2024-09-21T23:38:53.482480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"65'96","reported_seq":908,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481763+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:38:53.481763+0000","last_peered":"2024-09-21T23:38:53.481763+0000","last_clean":"2024-09-21T23:38:53.481763+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:38:53.481763+0000","last_undegraded":"2024-09-21T23:38:53.481763+0000","last_fullsized":"2024-09-21T23:38:53.481763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:02.495479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":16777270,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":894,"num_read_kb":43382,"num_write":81,"num_write_kb":49157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"67'5","reported_seq":73,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.257427+0000","last_change":"2024-09-21T23:38:24.109464+0000","last_active":"2024-09-21T23:38:54.257427+0000","last_peered":"2024-09-21T23:38:54.257427+0000","last_clean":"2024-09-21T23:38:54.257427+0000","last_became_active":"2024-09-21T23:38:22.074707+0000","last_became_peered":"2024-09-21T23:38:22.074707+0000","last_unstale":"2024-09-21T23:38:54.257427+0000","last_undegraded":"2024-09-21T23:38:54.257427+0000","last_fullsized":"2024-09-21T23:38:54.257427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:16.920782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021607010000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"68'85","reported_seq":505,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481198+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:38:53.481198+0000","last_peered":"2024-09-21T23:38:53.481198+0000","last_clean":"2024-09-21T23:38:53.481198+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:38:53.481198+0000","last_undegraded":"2024-09-21T23:38:53.481198+0000","last_fullsized":"2024-09-21T23:38:53.481198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:09:27.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874406,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":347,"num_read_kb":27166,"num_write":82,"num_write_kb":35842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.281886+0000","last_change":"2024-09-21T23:38:24.145472+0000","last_active":"2024-09-21T23:38:54.281886+0000","last_peered":"2024-09-21T23:38:54.281886+0000","last_clean":"2024-09-21T23:38:54.281886+0000","last_became_active":"2024-09-21T23:38:22.083188+0000","last_became_peered":"2024-09-21T23:38:22.083188+0000","last_unstale":"2024-09-21T23:38:54.281886+0000","last_undegraded":"2024-09-21T23:38:54.281886+0000","last_fullsized":"2024-09-21T23:38:54.281886+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:42.715435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065327500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'3","reported_seq":228,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481474+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:38:53.481474+0000","last_peered":"2024-09-21T23:38:53.481474+0000","last_clean":"2024-09-21T23:38:53.481474+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:38:53.481474+0000","last_undegraded":"2024-09-21T23:38:53.481474+0000","last_fullsized":"2024-09-21T23:38:53.481474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:22.504502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":2620,"num_write":3,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.257449+0000","last_change":"2024-09-21T23:38:24.108025+0000","last_active":"2024-09-21T23:38:54.257449+0000","last_peered":"2024-09-21T23:38:54.257449+0000","last_clean":"2024-09-21T23:38:54.257449+0000","last_became_active":"2024-09-21T23:38:22.084356+0000","last_became_peered":"2024-09-21T23:38:22.084356+0000","last_unstale":"2024-09-21T23:38:54.257449+0000","last_undegraded":"2024-09-21T23:38:54.257449+0000","last_fullsized":"2024-09-21T23:38:54.257449+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:07.570931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080679700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"58'41","reported_seq":309,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481228+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:38:53.481228+0000","last_peered":"2024-09-21T23:38:53.481228+0000","last_clean":"2024-09-21T23:38:53.481228+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:38:53.481228+0000","last_undegraded":"2024-09-21T23:38:53.481228+0000","last_fullsized":"2024-09-21T23:38:53.481228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:24:13.279673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":20971539,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":24652,"num_write":41,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"67'2","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.390073+0000","last_change":"2024-09-21T23:38:24.108228+0000","last_active":"2024-09-21T23:38:54.390073+0000","last_peered":"2024-09-21T23:38:54.390073+0000","last_clean":"2024-09-21T23:38:54.390073+0000","last_became_active":"2024-09-21T23:38:22.086357+0000","last_became_peered":"2024-09-21T23:38:22.086357+0000","last_unstale":"2024-09-21T23:38:54.390073+0000","last_undegraded":"2024-09-21T23:38:54.390073+0000","last_fullsized":"2024-09-21T23:38:54.390073+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:29.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498510000000001,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481279+0000","last_change":"2024-09-21T23:38:24.115041+0000","last_active":"2024-09-21T23:38:53.481279+0000","last_peered":"2024-09-21T23:38:53.481279+0000","last_clean":"2024-09-21T23:38:53.481279+0000","last_became_active":"2024-09-21T23:38:22.086737+0000","last_became_peered":"2024-09-21T23:38:22.086737+0000","last_unstale":"2024-09-21T23:38:53.481279+0000","last_undegraded":"2024-09-21T23:38:53.481279+0000","last_fullsized":"2024-09-21T23:38:53.481279+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:01:41.242218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057152899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"48'3","reported_seq":142,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.257656+0000","last_change":"2024-09-21T23:37:19.272996+0000","last_active":"2024-09-21T23:38:54.257656+0000","last_peered":"2024-09-21T23:38:54.257656+0000","last_clean":"2024-09-21T23:38:54.257656+0000","last_became_active":"2024-09-21T23:37:18.026611+0000","last_became_peered":"2024-09-21T23:37:18.026611+0000","last_unstale":"2024-09-21T23:38:54.257656+0000","last_undegraded":"2024-09-21T23:38:54.257656+0000","last_fullsized":"2024-09-21T23:38:54.257656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:48.223529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":11,"num_write":3,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"67'1","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.257425+0000","last_change":"2024-09-21T23:38:24.108086+0000","last_active":"2024-09-21T23:38:54.257425+0000","last_peered":"2024-09-21T23:38:54.257425+0000","last_clean":"2024-09-21T23:38:54.257425+0000","last_became_active":"2024-09-21T23:38:22.085324+0000","last_became_peered":"2024-09-21T23:38:22.085324+0000","last_unstale":"2024-09-21T23:38:54.257425+0000","last_undegraded":"2024-09-21T23:38:54.257425+0000","last_fullsized":"2024-09-21T23:38:54.257425+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:27.245291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078167000000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"68'130","reported_seq":506,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.482525+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:38:53.482525+0000","last_peered":"2024-09-21T23:38:53.482525+0000","last_clean":"2024-09-21T23:38:53.482525+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:38:53.482525+0000","last_undegraded":"2024-09-21T23:38:53.482525+0000","last_fullsized":"2024-09-21T23:38:53.482525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:18:36.282787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":20971574,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":710,"num_read_kb":19368,"num_write":196,"num_write_kb":37002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:53.481592+0000","last_change":"2024-09-21T23:38:24.185684+0000","last_active":"2024-09-21T23:38:53.481592+0000","last_peered":"2024-09-21T23:38:53.481592+0000","last_clean":"2024-09-21T23:38:53.481592+0000","last_became_active":"2024-09-21T23:38:22.085483+0000","last_became_peered":"2024-09-21T23:38:22.085483+0000","last_unstale":"2024-09-21T23:38:53.481592+0000","last_undegraded":"2024-09-21T23:38:53.481592+0000","last_fullsized":"2024-09-21T23:38:53.481592+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:36.524587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064978400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"68'81","reported_seq":312,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.257796+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:38:54.257796+0000","last_peered":"2024-09-21T23:38:54.257796+0000","last_clean":"2024-09-21T23:38:54.257796+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:38:54.257796+0000","last_undegraded":"2024-09-21T23:38:54.257796+0000","last_fullsized":"2024-09-21T23:38:54.257796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:46:50.102424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":20971596,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":218,"num_read_kb":24670,"num_write":97,"num_write_kb":45093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"67'1","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.281952+0000","last_change":"2024-09-21T23:38:24.145476+0000","last_active":"2024-09-21T23:38:54.281952+0000","last_peered":"2024-09-21T23:38:54.281952+0000","last_clean":"2024-09-21T23:38:54.281952+0000","last_became_active":"2024-09-21T23:38:22.084569+0000","last_became_peered":"2024-09-21T23:38:22.084569+0000","last_unstale":"2024-09-21T23:38:54.281952+0000","last_undegraded":"2024-09-21T23:38:54.281952+0000","last_fullsized":"2024-09-21T23:38:54.281952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:02.205745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060418199999999998,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"68'145","reported_seq":868,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T23:38:54.684726+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:38:54.684726+0000","last_peered":"2024-09-21T23:38:54.684726+0000","last_clean":"2024-09-21T23:38:54.684726+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:38:54.684726+0000","last_undegraded":"2024-09-21T23:38:54.684726+0000","last_fullsized":"2024-09-21T23:38:54.684726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:54.497706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1281,"num_read_kb":29761,"num_write":82,"num_write_kb":45063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":30,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":136315121,"num_objects":49,"num_object_clones":2,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":3720,"num_read_kb":171630,"num_write":585,"num_write_kb":265409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":272695296,"data_stored":285213154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":584,"ondisk_log_size":584,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607577,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91448,"kb_used_data":56100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94280392,"statfs":{"total":96636764160,"available":96543121408,"internally_reserved":0,"allocated":57446400,"data_stored":63218610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607578,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56616,"kb_used_data":21268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315224,"statfs":{"total":96636764160,"available":96578789376,"internally_reserved":0,"allocated":21778432,"data_stored":21275456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607577,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98168,"kb_used_data":62820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94273672,"statfs":{"total":96636764160,"available":96536240128,"internally_reserved":0,"allocated":64327680,"data_stored":63808978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607577,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79088,"kb_used_data":47900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292752,"statfs":{"total":96636764160,"available":96555778048,"internally_reserved":0,"allocated":49049600,"data_stored":54829964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607576,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89328,"kb_used_data":58136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282512,"statfs":{"total":96636764160,"available":96545292288,"internally_reserved":0,"allocated":59531264,"data_stored":59019355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607577,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57136,"kb_used_data":25944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314704,"statfs":{"total":96636764160,"available":96578256896,"internally_reserved":0,"allocated":26566656,"data_stored":26055247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25174016,"data_stored":25165878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58732544,"data_stored":58720351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48246784,"data_stored":54526044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":62930944,"data_stored":62914690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":56639488,"data_stored":62914671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":54,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:00.479 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [3, 1] 2024-09-21T23:39:00.479 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '3', '1'] 2024-09-21T23:39:00.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 3 1 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.086812444000003, 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:39:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:39:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:39:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.075553102000001, 2024-09-21T23:39:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:39:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:624 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.064757407000002, 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054719920000000002, 2024-09-21T23:39:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 624 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424343+0000", 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.424346+0000", 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425659+0000", 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013134100000000001 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425692+0000", 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3374000000000003e-05 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425712+0000", 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-05 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425838+0000", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126709 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.425983+0000", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014441200000000001 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.426190+0000", 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020753199999999999 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429353+0000", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031630629999999998 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429600+0000", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024618799999999999 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429747+0000", 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014790199999999999 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.429815+0000", 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8028999999999995e-05 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.053943967999999, 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:39:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:39:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:39:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.007848481000003, 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:39:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.995826393999998, 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:39:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:638 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:39:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.984347907, 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053659859999999997, 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 638 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504753+0000", 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.504755+0000", 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5339999999999998e-06 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505534+0000", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077814699999999997 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505566+0000", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2487000000000003e-05 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505582+0000", 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5393e-05 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505720+0000", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.505856+0000", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013566199999999999 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.506063+0000", 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206675 2024-09-21T23:39:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509744+0000", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003680872 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.509965+0000", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221179 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510054+0000", 2024-09-21T23:39:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8899000000000004e-05 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.510119+0000", 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5285999999999998e-05 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.974529453999999, 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.963710992000003, 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:39:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:39:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:684 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.711002731999997, 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055026119999999996, 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 684 2024-09-21T23:39:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778098+0000", 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.778100+0000", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779170+0000", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001069576 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779203+0000", 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3170000000000003e-05 2024-09-21T23:39:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779218+0000", 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5023e-05 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779343+0000", 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012489200000000001 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779493+0000", 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014961900000000001 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.779707+0000", 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021400499999999999 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783148+0000", 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034413740000000001 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783280+0000", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013202900000000001 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783491+0000", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021118499999999999 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.783601+0000", 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109536 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:688 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.690508573000002, 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055549559999999998, 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 688 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798592+0000", 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.798594+0000", 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799548+0000", 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095371900000000001 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799573+0000", 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-05 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799587+0000", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3499999999999999e-05 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799692+0000", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105036 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.799825+0000", 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013321200000000001 2024-09-21T23:39:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.800022+0000", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019666200000000001 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803718+0000", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003696179 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.803857+0000", 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013872600000000001 2024-09-21T23:39:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804047+0000", 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019052500000000001 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.804147+0000", 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9943000000000002e-05 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:690 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.680091863999998, 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054051699999999999, 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 690 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809009+0000", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809011+0000", 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1600000000000001e-06 2024-09-21T23:39:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809825+0000", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081403400000000005 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809857+0000", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2175999999999998e-05 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.809873+0000", 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5207e-05 2024-09-21T23:39:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810002+0000", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012908900000000001 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810137+0000", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013584900000000001 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.810346+0000", 2024-09-21T23:39:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020878800000000001 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.813922+0000", 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035753099999999999 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814169+0000", 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024718899999999998 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814314+0000", 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014533800000000001 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.814414+0000", 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010003 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.246600430999997, 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.246507602000001, 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.909683453, 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:39:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:39:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.88853108, 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:39:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:39:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:39:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.302300249000002, 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:39:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:39:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.166970720999998, 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:128 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.962204281999998, 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054825550000000001, 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 128 2024-09-21T23:39:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526897+0000", 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.526898+0000", 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.906e-06 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527948+0000", 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010493989999999999 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527974+0000", 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6407e-05 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.527988+0000", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3720000000000001e-05 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528089+0000", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100782 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528224+0000", 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135649 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.528433+0000", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000208982 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.531670+0000", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032363919999999998 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532210+0000", 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053980599999999999 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532309+0000", 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9335999999999999e-05 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.532379+0000", 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0176e-05 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.941921757999999, 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:39:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:00.600 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:00.603 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:00.601+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:00.702 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:00.701+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:00.761 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:00.761 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:00.803 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:00.804+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:00.904 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:00.901+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:00.935 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:00.935 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:00.935 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:00.935 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:00.935 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:00.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:01.045 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:01.045+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:01.105 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:01.105+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.820777120000002, 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:39:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:39:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.819913546999999, 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:39:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:39:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.819251035999997, 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.136 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.818576966999998, 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:39:01.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:39:01.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.817850386000003, 2024-09-21T23:39:01.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:39:01.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.817124765000003, 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:39:01.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:39:01.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:39:01.150 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.816387792999997, 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:39:01.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:39:01.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:652 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'27, mlcod=54'25)", 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.538639478, 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063874020000000004, 2024-09-21T23:39:01.155 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583268+0000", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584341+0000", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072999 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584369+0000", 2024-09-21T23:39:01.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7112999999999999e-05 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584382+0000", 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3672999999999999e-05 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584481+0000", 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8655999999999999e-05 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584516+0000", 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589566+0000", 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050499890000000004 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589654+0000", 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8598999999999997e-05 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:654 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'29, mlcod=54'27)", 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.524520197000001, 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005678677, 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:01.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597388+0000", 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.72e-06 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598151+0000", 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076301400000000001 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598181+0000", 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9972999999999999e-05 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598195+0000", 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4059999999999999e-05 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598345+0000", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015011199999999999 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598411+0000", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5560000000000002e-05 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.602957+0000", 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045466150000000004 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.603065+0000", 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107623 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:656 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'31, mlcod=54'29)", 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.511763443, 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062273229999999999, 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610145+0000", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610983+0000", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083891 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611010+0000", 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6927000000000002e-05 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611038+0000", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7915999999999998e-05 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611167+0000", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128906 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611226+0000", 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8423e-05 2024-09-21T23:39:01.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616286+0000", 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050602449999999997 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616370+0000", 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4486000000000004e-05 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:694 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.287438807000001, 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056719939999999996, 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834469+0000", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-06 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835363+0000", 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089360600000000002 2024-09-21T23:39:01.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835404+0000", 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0630000000000002e-05 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835424+0000", 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0123e-05 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835590+0000", 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000166625 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835656+0000", 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5510000000000001e-05 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840062+0000", 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044064239999999999 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840140+0000", 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7279000000000003e-05 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:696 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.274773048, 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065145940000000003, 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847135+0000", 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848041+0000", 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090629299999999996 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848068+0000", 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7453e-05 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848082+0000", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3407000000000001e-05 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848168+0000", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6229000000000004e-05 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848203+0000", 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4480000000000002e-05 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853578+0000", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053756660000000003 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853648+0000", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9733000000000005e-05 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:01.177 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.175757830999999, 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:01.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.181 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.161984255, 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:39:01.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.099805852000003, 2024-09-21T23:39:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:39:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:39:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:105 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'54, mlcod=58'52)", 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.720726059, 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059755880000000004, 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401182+0000", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402099+0000", 2024-09-21T23:39:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091730900000000005 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402122+0000", 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402139+0000", 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6976000000000001e-05 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402233+0000", 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4480000000000003e-05 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402263+0000", 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9522999999999998e-05 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407054+0000", 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047915930000000002 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407156+0000", 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101743 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:107 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'56, mlcod=58'54)", 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.708530668000002, 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055358780000000002, 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413377+0000", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414193+0000", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081642699999999999 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414216+0000", 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2286999999999999e-05 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414233+0000", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7119999999999999e-05 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414372+0000", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414405+0000", 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3673000000000001e-05 2024-09-21T23:39:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418791+0000", 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043853770000000002 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418912+0000", 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121025 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:137 2.3 e58/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 58'58, mlcod=58'56)", 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.548298691999999, 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060185819999999998, 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573609+0000", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4899999999999999e-06 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574550+0000", 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094037000000000005 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574573+0000", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3516999999999999e-05 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574585+0000", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1712999999999999e-05 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574693+0000", 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010851199999999999 2024-09-21T23:39:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574725+0000", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.137e-05 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579522+0000", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047972780000000003 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579626+0000", 2024-09-21T23:39:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104332 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:170 2.3 e58/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 58'72, mlcod=58'70)", 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.383648635, 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057334100000000004, 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738260+0000", 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7e-06 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739118+0000", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085804700000000002 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739147+0000", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9343e-05 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739161+0000", 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3627e-05 2024-09-21T23:39:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739247+0000", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6779000000000004e-05 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739280+0000", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3186999999999999e-05 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743866+0000", 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045857509999999999 2024-09-21T23:39:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743991+0000", 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124976 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:175 2.3 e58/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 58'76, mlcod=58'74)", 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.357500665, 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061086780000000002, 2024-09-21T23:39:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764407+0000", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.637e-06 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765253+0000", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084597600000000002 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765283+0000", 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.915e-05 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765298+0000", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5444e-05 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765398+0000", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9622000000000003e-05 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765430+0000", 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2867000000000002e-05 2024-09-21T23:39:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770401+0000", 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049709890000000003 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770514+0000", 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011299299999999999 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:01.213 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:01.214 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:01.214 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:01.212+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:01.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:01.314 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:01.313+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:01.345 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:01.346 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:01.346 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:01.346 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:01.359 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:01.399 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:01.399 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:01.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:01.415 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:01.416 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:01.416+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:01.433 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:01.433 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:01.453 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:01.453 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:01.473 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:01.474 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:01.478 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:01.478 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:01.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:01.494 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:01.494 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:set 3.2 pg_upmap mapping to [3,1] 2024-09-21T23:39:01.516 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:01.513+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:01.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:39:01.617 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:01.616+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:01.660 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:01.660 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:01.674 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:01.718 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:01.717+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:01.819 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:01.817+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:01.840 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:01.840 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:01.854 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:01.919 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:01.917+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:01.978 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:01.978 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:01.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:02.035 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:02.036+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:02.109 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.109 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.121 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:02.120+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:02.123 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:02.221 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:02.221+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:02.291 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.292 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.306 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:02.356 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:02.356+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:02.428 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:02.424+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:02.430 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.430 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.444 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:02.524 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:02.521+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:02.531 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:39:02.617 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.618 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.624 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:02.621+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:02.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:02.727 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:02.725+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:02.765 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.765 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:02.826 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:02.828+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:02.926 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:02.925+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:02.940 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:02.940 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:02.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:03.027 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:03.028+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:03.122 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:03.122 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:03.128 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:03.128+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:03.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:03.229 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:03.229+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:03.256 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:03.257 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:03.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:03.330 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:03.332+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:03.430 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:03.429+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:03.431 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:03.431 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:03.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:03.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:03.531 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:03.528+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:03.612 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:03.613 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:03.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:03.632 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:03.632+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:03.674 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:03.674 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:03.690 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:03.691 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:03.709 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:03.710 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:03.728 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:03.729 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:03.732 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:03.729+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:03.746 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:03.746 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:03.746 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:03.746 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:03.749 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:03.749 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:03.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:03.770 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:03.771 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:03.833 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:03.832+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:03.884 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:03.884 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:03.884 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:03.884 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:03.884 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:03.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:03.945 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:03.945+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:04.010 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.709602830000001, 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:39:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:39:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:39:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.708739256999998, 2024-09-21T23:39:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.708076746000003, 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.707402676999997, 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:39:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.706676096000002, 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:39:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:39:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:39:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.705950475000002, 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:39:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:39:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.705213503000003, 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:39:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:652 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'27, mlcod=54'25)", 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.427465187999999, 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063874020000000004, 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583267+0000", 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.583268+0000", 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584341+0000", 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072999 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584369+0000", 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7112999999999999e-05 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584382+0000", 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3672999999999999e-05 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584481+0000", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8655999999999999e-05 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.584516+0000", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4943000000000001e-05 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589566+0000", 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050499890000000004 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.589654+0000", 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8598999999999997e-05 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:654 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'29, mlcod=54'27)", 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.413345907, 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005678677, 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597386+0000", 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.597388+0000", 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.72e-06 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598151+0000", 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076301400000000001 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598181+0000", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9972999999999999e-05 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598195+0000", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4059999999999999e-05 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598345+0000", 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015011199999999999 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.598411+0000", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5560000000000002e-05 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.602957+0000", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045466150000000004 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.603065+0000", 2024-09-21T23:39:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107623 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:656 2.0 e54/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 54'31, mlcod=54'29)", 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.400589152999999, 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062273229999999999, 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610143+0000", 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610145+0000", 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.610983+0000", 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083891 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611010+0000", 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6927000000000002e-05 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611038+0000", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7915999999999998e-05 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611167+0000", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128906 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.611226+0000", 2024-09-21T23:39:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8423e-05 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616286+0000", 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050602449999999997 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.616370+0000", 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4486000000000004e-05 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:694 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.176264517, 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056719939999999996, 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834468+0000", 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.834469+0000", 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-06 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835363+0000", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089360600000000002 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835404+0000", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0630000000000002e-05 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835424+0000", 2024-09-21T23:39:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0123e-05 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835590+0000", 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000166625 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.835656+0000", 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5510000000000001e-05 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840062+0000", 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044064239999999999 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.840140+0000", 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7279000000000003e-05 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:696 2.3 e54/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.163598757999999, 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065145940000000003, 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847133+0000", 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.847135+0000", 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.333e-06 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848041+0000", 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090629299999999996 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848068+0000", 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7453e-05 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848082+0000", 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3407000000000001e-05 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848168+0000", 2024-09-21T23:39:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6229000000000004e-05 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.848203+0000", 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4480000000000002e-05 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853578+0000", 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053756660000000003 2024-09-21T23:39:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.853648+0000", 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9733000000000005e-05 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.064583540999998, 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.050809964999999, 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:39:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 35.988631562000002, 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:39:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:105 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'54, mlcod=58'52)", 2024-09-21T23:39:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.609551768999999, 2024-09-21T23:39:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059755880000000004, 2024-09-21T23:39:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401180+0000", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.401182+0000", 2024-09-21T23:39:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.217e-06 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402099+0000", 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091730900000000005 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402122+0000", 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2747000000000001e-05 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402139+0000", 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6976000000000001e-05 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402233+0000", 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4480000000000003e-05 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.402263+0000", 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9522999999999998e-05 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407054+0000", 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047915930000000002 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.407156+0000", 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101743 2024-09-21T23:39:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:107 2.3 e58/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 58'56, mlcod=58'54)", 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.597356378000001, 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055358780000000002, 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413376+0000", 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.413377+0000", 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414193+0000", 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081642699999999999 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414216+0000", 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2286999999999999e-05 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414233+0000", 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7119999999999999e-05 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414372+0000", 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013885199999999999 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.414405+0000", 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3673000000000001e-05 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418791+0000", 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043853770000000002 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.418912+0000", 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121025 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:137 2.3 e58/17 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head v 58'58, mlcod=58'56)", 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.437124401999998, 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060185819999999998, 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573608+0000", 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.573609+0000", 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4899999999999999e-06 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574550+0000", 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094037000000000005 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574573+0000", 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3516999999999999e-05 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574585+0000", 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1712999999999999e-05 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574693+0000", 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010851199999999999 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.574725+0000", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.137e-05 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579522+0000", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047972780000000003 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.579626+0000", 2024-09-21T23:39:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104332 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:170 2.3 e58/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 58'72, mlcod=58'70)", 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.272474344999999, 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057334100000000004, 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738258+0000", 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.738260+0000", 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7e-06 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739118+0000", 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085804700000000002 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739147+0000", 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9343e-05 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739161+0000", 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3627e-05 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739247+0000", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6779000000000004e-05 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.739280+0000", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3186999999999999e-05 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743866+0000", 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045857509999999999 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.743991+0000", 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124976 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:175 2.3 e58/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 58'76, mlcod=58'74)", 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.246326374999999, 2024-09-21T23:39:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061086780000000002, 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764406+0000", 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.764407+0000", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.637e-06 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765253+0000", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00084597600000000002 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765283+0000", 2024-09-21T23:39:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.915e-05 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765298+0000", 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5444e-05 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765398+0000", 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9622000000000003e-05 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.765430+0000", 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2867000000000002e-05 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770401+0000", 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049709890000000003 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.770514+0000", 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011299299999999999 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:04.115 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:04.116 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:04.117 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:04.116+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:04.217 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:04.217+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:04.248 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:04.249 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:04.249 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:04.249 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:04.263 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:04.318 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:04.316+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:04.384 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:04.384 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:04.384 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:04.385 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:04.385 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:04.398 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:04.439 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:04.440+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:04.519 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:04.517+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:04.548 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:04.548 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23674\b' 2024-09-21T23:39:04.548 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23674\b' 2024-09-21T23:39:04.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.563 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.33445366399999, 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:39:04.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.987148421000001, 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.570 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:39:04.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:39:04.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.987068988000001, 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:39:04.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:39:04.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.987037985000001, 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.580 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:39:04.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:39:04.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.986993427000002, 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:39:04.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:39:04.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.986958622, 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.588 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:39:04.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:39:04.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.963345399000001, 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:39:04.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:39:04.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.963310483000001, 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:39:04.598 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:39:04.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.963278733999999, 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:39:04.604 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:39:04.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:39:04.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.610 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.963239073, 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:39:04.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:39:04.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.963207833999999, 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:39:04.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:39:04.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.622 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.962753457000002, 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:39:04.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:39:04.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.962379273, 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:39:04.629 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:04.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:39:04.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.636 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.960121864000001, 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.955778178999999, 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.643 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:39:04.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.949614556, 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.946384661, 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:39:04.655 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:39:04.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:39:04.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:39:04.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.945998405000001, 2024-09-21T23:39:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:04.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:39:04.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:04.667 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.936237025000001, 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:39:04.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:39:04.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.936007768, 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:39:04.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:04.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:39:04.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:04.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:39:04.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:39:04.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:04.680 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23674\b' 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23674\b' 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:04.682 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:04.683 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:04.683 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:04.680+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:04.782 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:04.781+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:04.844 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:04.844 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:04.844 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:04.845 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:04.858 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:04.904 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:04.904+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:04.974 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:04.974 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:04.974 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:04.974 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:04.974 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:04.984 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:04.984+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:04.988 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:05.085 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:05.085+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:05.125 INFO:tasks.workunit.client.0.smithi165.stderr:+ LOG_FILE=/tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:05.125 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n '' ']' 2024-09-21T23:39:05.125 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map --quiesce rbd/testrbdnbd21555 --log-file=/tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:05.125 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:05.126 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:05.127 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:05.127 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:05.130 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:05.130 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:05.130 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map --quiesce rbd/testrbdnbd21555 --log-file=/tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.797203271999997, 2024-09-21T23:39:05.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:39:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.785539233000002, 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:39:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.773973003999998, 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:39:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:39:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.288035467999997, 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:39:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:702 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.280006505999999, 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002866164, 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880718+0000", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.880719+0000", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2410000000000001e-06 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881020+0000", 2024-09-21T23:39:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000301341 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881038+0000", 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7932e-05 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881049+0000", 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0523000000000001e-05 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881122+0000", 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3595000000000003e-05 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.881153+0000", 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0102000000000001e-05 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883546+0000", 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023938890000000002 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.883584+0000", 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7540999999999999e-05 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:704 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'34, mlcod=55'32)", 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.271970318000001, 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027588729999999998, 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888754+0000", 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.888755+0000", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.093e-06 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889090+0000", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033504800000000001 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889109+0000", 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8627999999999999e-05 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889119+0000", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7219999999999994e-06 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889189+0000", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0308999999999994e-05 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.889221+0000", 2024-09-21T23:39:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1755999999999999e-05 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891424+0000", 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022030650000000001 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.891513+0000", 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9252000000000005e-05 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.264125350999997, 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.255634290000003, 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:39:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:710 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'40, mlcod=55'38)", 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.245094377999997, 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002832621, 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:39:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915630+0000", 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.915631+0000", 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.158e-06 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916242+0000", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061049200000000002 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916260+0000", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7938e-05 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916268+0000", 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5590000000000006e-06 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916328+0000", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0189999999999998e-05 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.916350+0000", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2201e-05 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918425+0000", 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002074944 2024-09-21T23:39:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.918463+0000", 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7138999999999998e-05 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.236498705999999, 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:39:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:39:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.227094385999997, 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:39:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:39:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.581659260000002, 2024-09-21T23:39:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:39:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.550004876999999, 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:39:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.546301802999999, 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:39:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.990698374000001, 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:39:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.532151122999998, 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:39:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:39:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:39:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.520787864999999, 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:39:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.509392907999999, 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:39:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:39:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:39:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.499784169000002, 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:39:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5487.0:6 3.3 3:daeef835:::rbd_id.testrbdnbd21555:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.631303379999999, 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027352959999999999, 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5487", 2024-09-21T23:39:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2667436824", 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 6 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529421+0000", 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529423+0000", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7400000000000001e-06 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529434+0000", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1643e-05 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529457+0000", 2024-09-21T23:39:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2996000000000001e-05 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529479+0000", 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1953000000000001e-05 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529629+0000", 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000149729 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.529878+0000", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024851400000000003 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.530224+0000", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034664199999999998 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.531634+0000", 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001410288 2024-09-21T23:39:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.531877+0000", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024207100000000001 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.532034+0000", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015757299999999999 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:47.532156+0000", 2024-09-21T23:39:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012214699999999999 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.253 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:05.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:05.255 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:05.253+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:05.276 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd1 2024-09-21T23:39:05.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:05.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:05.277 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:05.277 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:05.277 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:05.355 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:05.352+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:05.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23854 2024-09-21T23:39:05.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23854 2024-09-21T23:39:05.366 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23854 -C rbd-nbd 2024-09-21T23:39:05.374 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:05.375 INFO:tasks.workunit.client.0.smithi165.stdout: 23854 ? 00:00:00 rbd-nbd 2024-09-21T23:39:05.375 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo mkfs /dev/nbd1 2024-09-21T23:39:05.375 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:05.375 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:05.377 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:05.377 INFO:tasks.workunit.client.0.smithi165.stderr:++ which mkfs 2024-09-21T23:39:05.378 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/mkfs 2024-09-21T23:39:05.379 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:05.379 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/mkfs /dev/nbd1 2024-09-21T23:39:05.379 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:05.379 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:05.380 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:05.380 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:05.394 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:05.484 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:05.484+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:05.494 INFO:tasks.workunit.client.0.smithi165.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-21T23:39:05.537 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:39:05.557 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:05.557+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:05.573 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:05.573 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:05.574 INFO:tasks.workunit.client.0.smithi165.stdout:Discarding device blocks: 0/16384 done 2024-09-21T23:39:05.574 INFO:tasks.workunit.client.0.smithi165.stdout:Creating filesystem with 16384 4k blocks and 16384 inodes 2024-09-21T23:39:05.574 INFO:tasks.workunit.client.0.smithi165.stdout: 2024-09-21T23:39:05.574 INFO:tasks.workunit.client.0.smithi165.stdout:Allocating group tables: 0/1 done 2024-09-21T23:39:05.591 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:05.597 INFO:tasks.workunit.client.0.smithi165.stdout:Writing inode tables: 0/1 done 2024-09-21T23:39:05.636 INFO:tasks.workunit.client.0.smithi165.stdout:Writing superblocks and filesystem accounting information: 0/1 done 2024-09-21T23:39:05.636 INFO:tasks.workunit.client.0.smithi165.stdout: 2024-09-21T23:39:05.638 INFO:tasks.workunit.client.0.smithi165.stderr:+ mkdir /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:05.657 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo mount /dev/nbd1 /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:05.657 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:05.658 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:05.657+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:05.658 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:05.659 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:05.659 INFO:tasks.workunit.client.0.smithi165.stderr:++ which mount 2024-09-21T23:39:05.661 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/mount 2024-09-21T23:39:05.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:05.662 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/mount /dev/nbd1 /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:05.699 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd snap create rbd/testrbdnbd21555@quiesce3 2024-09-21T23:39:05.758 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:05.758 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:05.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.660386867, 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:39:05.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:39:05.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.64630932, 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:39:05.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:39:05.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:39:05.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.637217509000003, 2024-09-21T23:39:05.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:39:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:39:05.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:05.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.615170413000001, 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:05.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:39:05.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.511795081000002, 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:39:05.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:39:05.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:39:05.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.511665942999997, 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:39:05.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:39:05.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.479861526000001, 2024-09-21T23:39:05.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:39:05.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:34 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.23707241400000001, 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023817232000000001, 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 34 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518869+0000", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518884+0000", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4742e-05 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518940+0000", 2024-09-21T23:39:05.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5745999999999998e-05 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518996+0000", 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5943999999999998e-05 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519161+0000", 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016499700000000001 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519412+0000", 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025155899999999999 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519930+0000", 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051799400000000003 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.537789+0000", 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017859223 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542572+0000", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004782171 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542635+0000", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3083999999999994e-05 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542684+0000", 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9135999999999998e-05 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.801 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:36 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.23687883700000001, 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024223537999999999, 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.802 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519061+0000", 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.283e-06 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519069+0000", 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7190000000000001e-06 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519094+0000", 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4904000000000001e-05 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519111+0000", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7251000000000001e-05 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529102+0000", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099906300000000003 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529230+0000", 2024-09-21T23:39:05.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012755500000000001 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529483+0000", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025336899999999998 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.539134+0000", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096515980000000008 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542876+0000", 2024-09-21T23:39:05.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003741439 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543167+0000", 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029152799999999999 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543284+0000", 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011626200000000001 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:59 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.18778104800000001, 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018089152000000001, 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 59 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568160+0000", 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9589999999999998e-06 2024-09-21T23:39:05.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568169+0000", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0129999999999999e-06 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568192+0000", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-05 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568207+0000", 2024-09-21T23:39:05.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5804e-05 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568324+0000", 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116344 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568338+0000", 2024-09-21T23:39:05.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4154e-05 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571148+0000", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028103519999999999 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571230+0000", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-05 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571406+0000", 2024-09-21T23:39:05.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017574999999999999 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576139+0000", 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047326759999999999 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586043+0000", 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099038500000000005 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586151+0000", 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108323 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586247+0000", 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6277e-05 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:92 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2125824~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.147957699, 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015425227, 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 92 2024-09-21T23:39:05.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607982+0000", 2024-09-21T23:39:05.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4900000000000001e-07 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608111+0000", 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012953800000000001 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608118+0000", 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5180000000000001e-06 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608121+0000", 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4989999999999999e-06 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614540+0000", 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064183850000000004 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614587+0000", 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7521000000000001e-05 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614665+0000", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7497999999999996e-05 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617185+0000", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025206109999999999 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623310+0000", 2024-09-21T23:39:05.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006124694 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623355+0000", 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4842000000000002e-05 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623407+0000", 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1771999999999998e-05 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:93 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2256896~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.14781186700000001, 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015447901, 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 93 2024-09-21T23:39:05.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.41e-07 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608256+0000", 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128291 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608264+0000", 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7740000000000004e-06 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608269+0000", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2279999999999998e-06 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615063+0000", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067938929999999996 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615159+0000", 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6428999999999994e-05 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615288+0000", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012889800000000001 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617455+0000", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021670169999999998 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623474+0000", 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060186069999999996 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623520+0000", 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6347999999999997e-05 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623575+0000", 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5074999999999997e-05 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:94 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2387968~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.14765904599999999, 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017672687999999999, 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 94 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608281+0000", 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7800000000000001e-07 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608423+0000", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142819 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608433+0000", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.38e-06 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608440+0000", 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615922+0000", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074819919999999998 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615968+0000", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6622000000000002e-05 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616051+0000", 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2434000000000004e-05 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617461+0000", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014100060000000001 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625814+0000", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083532390000000002 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625896+0000", 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1805999999999996e-05 2024-09-21T23:39:05.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625953+0000", 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6932000000000003e-05 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:95 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2519040~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.147488126, 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017624719, 2024-09-21T23:39:05.832 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608452+0000", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608524+0000", 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2565999999999996e-05 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608532+0000", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2879999999999993e-06 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608539+0000", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616439+0000", 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007900041 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616540+0000", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010035999999999999 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616676+0000", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0001362 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619529+0000", 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028531250000000002 2024-09-21T23:39:05.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625999+0000", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064702960000000004 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626029+0000", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9702000000000001e-05 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626076+0000", 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6656000000000002e-05 2024-09-21T23:39:05.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:96 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2650112~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.147386137, 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017659136999999998, 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608620+0000", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6256999999999999e-05 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608634+0000", 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608640+0000", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1659999999999999e-06 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617495+0000", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088556810000000007 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617544+0000", 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8162999999999999e-05 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617618+0000", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4759999999999996e-05 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619535+0000", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001916374 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626121+0000", 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065866299999999996 2024-09-21T23:39:05.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626166+0000", 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4202999999999997e-05 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626212+0000", 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6434000000000003e-05 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:97 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2781184~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.14728894200000001, 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017675217999999999, 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 97 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608651+0000", 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9299999999999998e-07 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608730+0000", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9598000000000006e-05 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608739+0000", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0119999999999994e-06 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608746+0000", 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2349999999999996e-06 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618001+0000", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092555099999999998 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618092+0000", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0552999999999999e-05 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618235+0000", 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014355499999999999 2024-09-21T23:39:05.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620378+0000", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021429750000000001 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626257+0000", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0058784980000000002 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626283+0000", 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.601e-05 2024-09-21T23:39:05.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626325+0000", 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2679000000000003e-05 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:98 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2912256~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.147183387, 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017700770000000001, 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.849 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 98 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608873+0000", 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117304 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608887+0000", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3386e-05 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608894+0000", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618777+0000", 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098831249999999995 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618825+0000", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8563000000000002e-05 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618902+0000", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6130000000000005e-05 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621131+0000", 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022290230000000001 2024-09-21T23:39:05.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626371+0000", 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052399329999999996 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626412+0000", 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1844999999999997e-05 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626456+0000", 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4013e-05 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:99 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3043328~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.147033724, 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017665733999999999, 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 99 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608906+0000", 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608993+0000", 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6874999999999997e-05 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609004+0000", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0849999999999999e-05 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609010+0000", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619684+0000", 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010673867 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619760+0000", 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6249999999999997e-05 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619880+0000", 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011974200000000001 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622034+0000", 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021542639999999999 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626501+0000", 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044669829999999999 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626528+0000", 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626571+0000", 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3123e-05 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:100 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3174400~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.14688542199999999, 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01765369, 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:05.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609143+0000", 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9022000000000002e-05 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609153+0000", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8160000000000005e-06 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609161+0000", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3420000000000008e-06 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620487+0000", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011325052 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620536+0000", 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654999999999998e-05 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620615+0000", 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8403000000000001e-05 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622168+0000", 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015539270000000001 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626619+0000", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044508730000000002 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626662+0000", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2175000000000002e-05 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626707+0000", 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5741999999999997e-05 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:101 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3305472~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.14676472199999999, 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020329684000000001, 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:05.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:05.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609175+0000", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2200000000000004e-07 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609264+0000", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8688000000000002e-05 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609273+0000", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0380000000000005e-06 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:05.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609277+0000", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.036e-06 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621290+0000", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012013429000000001 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621385+0000", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5260000000000006e-05 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:05.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621524+0000", 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013853100000000001 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622385+0000", 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00086160000000000002 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629426+0000", 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007040129 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629455+0000", 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9274000000000001e-05 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629504+0000", 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9076999999999998e-05 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:05.872 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:05.873 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:05.756+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:05.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:05.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:05.916 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:05.916+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:05.976 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:05.977+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:06.005 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:06.005 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:06.005 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:06.005 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:06.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:06.023 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:06.023 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:06.040 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:06.040 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:06.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:06.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:06.077 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:06.077+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:06.079 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:06.079 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:06.099 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:06.099 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:06.120 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:06.120 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:06.137 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.138 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:06.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:06.178 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:06.176+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:06.279 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:06.277+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:06.314 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.314 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:06.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:06.380 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:06.380+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:06.481 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:06.484+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:06.490 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.490 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:06.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:06.532 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:39:06.532 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:39:06.533 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T23:39:06.533 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:39:06.533 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:39:06.533 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T23:39:06.568 INFO:tasks.workunit.client.0.smithi165.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-09-21T23:39:06.576 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/tmp/tmp.h0CUjCtN4b/mnt/test bs=1M count=1 oflag=direct 2024-09-21T23:39:06.576 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:06.576 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:06.577 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:06.577 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:39:06.579 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:39:06.579 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:06.579 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/tmp/tmp.h0CUjCtN4b/mnt/test bs=1M count=1 oflag=direct 2024-09-21T23:39:06.581 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:06.581+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:06.593 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:39:06.593 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:39:06.593 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00231686 s, 453 MB/s 2024-09-21T23:39:06.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo umount /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:06.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:06.595 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:06.596 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:06.596 INFO:tasks.workunit.client.0.smithi165.stderr:++ which umount 2024-09-21T23:39:06.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/umount 2024-09-21T23:39:06.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:06.598 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/umount /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:06.623 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd1 23854 2024-09-21T23:39:06.623 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd1 2024-09-21T23:39:06.623 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=23854 2024-09-21T23:39:06.624 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:06.624 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:06.624 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:06.624 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:06.624 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:06.626 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:06.626 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:06.626 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:06.630 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.630 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:06.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:06.667 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:06.668 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "" 2024-09-21T23:39:06.668 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:06.689 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:06.689+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:06.737 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:06.737 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^23854\b' 2024-09-21T23:39:06.737 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^23854\b' 2024-09-21T23:39:06.783 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:06.781+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:06.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:06.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:06.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *23854\b' 2024-09-21T23:39:06.796 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *23854\b' 2024-09-21T23:39:06.804 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:06.804 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:06.812 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.812 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:06.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:06.884 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:06.883+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:06.985 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:06.983+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:06.990 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:06.991 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:07.086 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:07.085+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:07.127 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.128 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:07.187 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:07.183+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:07.288 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:07.287+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:07.302 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.302 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.305 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:07.305 INFO:tasks.workunit.client.0.smithi165.stderr:+ cat /tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:07.306 INFO:tasks.workunit.client.0.smithi165.stdout:2024-09-21T23:39:05.218+0000 7fe1a0f84a80 0 ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable), process rbd-nbd, pid 23845 2024-09-21T23:39:07.306 INFO:tasks.workunit.client.0.smithi165.stdout:2024-09-21T23:39:05.218+0000 7fe1a0f84a80 0 pidfile_write: ignore empty --pid-file 2024-09-21T23:39:07.307 INFO:tasks.workunit.client.0.smithi165.stdout:2024-09-21T23:39:05.594+0000 7fe19f277640 0 -- 172.21.15.165:0/3546672560 >> [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] conn(0x55ec0a659810 msgr2=0x55ec0a65bc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-21T23:39:07.307 INFO:tasks.workunit.client.0.smithi165.stdout:2024-09-21T23:39:05.614+0000 7fe19f277640 0 -- 172.21.15.165:0/3546672560 >> [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] conn(0x7fe18403b860 msgr2=0x7fe18403dc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:39:07.307 INFO:tasks.workunit.client.0.smithi165.stdout:2024-09-21T23:39:06.714+0000 7fe17b7fe640 0 rbd-nbd: disconnect request received 2024-09-21T23:39:07.307 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep 'quiesce failed' /tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:07.307 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep 'quiesce failed' /tmp/tmp.h0CUjCtN4b/rbd-nbd.log 2024-09-21T23:39:07.308 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:07.309 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd --show-cookie map rbd/testrbdnbd21555 2024-09-21T23:39:07.309 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:07.309 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:07.311 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:07.311 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:07.313 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:07.313 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:07.313 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie map rbd/testrbdnbd21555 2024-09-21T23:39:07.316 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:07.389 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:07.389+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:07.450 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.450 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ OUT='/dev/nbd1 577c016b-6ffd-4f9f-8208-9ca7afd5e421' 2024-09-21T23:39:07.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ read DEV COOKIE 2024-09-21T23:39:07.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:07.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:07.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:07.458 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:07.458 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:07.464 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:07.512 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:07.511+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:07.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=23970 2024-09-21T23:39:07.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 23970 2024-09-21T23:39:07.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 23970 -C rbd-nbd 2024-09-21T23:39:07.555 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:07.555 INFO:tasks.workunit.client.0.smithi165.stdout: 23970 ? 00:00:00 rbd-nbd 2024-09-21T23:39:07.556 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo mount /dev/nbd1 /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:07.556 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:07.556 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:07.557 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:07.558 INFO:tasks.workunit.client.0.smithi165.stderr:++ which mount 2024-09-21T23:39:07.559 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/mount 2024-09-21T23:39:07.560 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:07.560 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/mount /dev/nbd1 /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:07.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:39:07.595 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.595 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.610 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device detach rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.610 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:07.610 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:07.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:07.612 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:07.612 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:07.613 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:07.613 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:07.613 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device detach rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.617 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:07.617+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:07.692 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:07.689+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:07.723 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false get_pid rbd 2024-09-21T23:39:07.723 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:07.723 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:07.724 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:07.724 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:07.724 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.782 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.792 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID= 2024-09-21T23:39:07.792 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n '' 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 1 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo rbd device attach --device /dev/nbd1 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device attach --device /dev/nbd1 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:07.793 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:07.793 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:07.793+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:07.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:07.794 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:07.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:07.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:07.795 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.798 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:07.827 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: could not validate attach request 2024-09-21T23:39:07.827 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: mismatching the image and the device may lead to data corruption 2024-09-21T23:39:07.828 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: must specify --cookie or --force to proceed 2024-09-21T23:39:07.830 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n 577c016b-6ffd-4f9f-8208-9ca7afd5e421 ']' 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie 577c016b-6ffd-4f9f-8208-9ca7afd5e421 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:07.831 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:07.833 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:07.833 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:07.833 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie 577c016b-6ffd-4f9f-8208-9ca7afd5e421 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:07.893 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:07.891+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:07.921 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:07.922 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:07.935 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:07.954 INFO:tasks.workunit.client.0.smithi165.stdout:/dev/nbd1 2024-09-21T23:39:07.961 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:07.961 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:07.961 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:07.962 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:07.962 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:07.994 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:07.993+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:08.041 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24040 2024-09-21T23:39:08.041 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24040 2024-09-21T23:39:08.041 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24040 -C rbd-nbd 2024-09-21T23:39:08.046 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:08.047 INFO:tasks.workunit.client.0.smithi165.stdout: 24040 ? 00:00:00 rbd-nbd 2024-09-21T23:39:08.047 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device detach /dev/nbd1 --device-type nbd 2024-09-21T23:39:08.047 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:08.048 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:08.049 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:08.049 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:08.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:08.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:08.051 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device detach /dev/nbd1 --device-type nbd 2024-09-21T23:39:08.094 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:08.093+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:08.104 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:08.104 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:08.120 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:08.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:08.147 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false get_pid rbd 2024-09-21T23:39:08.147 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:08.147 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:08.147 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:08.148 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:08.148 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:08.195 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:08.191+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID= 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n '' 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 1 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n 577c016b-6ffd-4f9f-8208-9ca7afd5e421 ']' 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie 577c016b-6ffd-4f9f-8208-9ca7afd5e421 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:08.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:08.217 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:08.217 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:08.217 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:08.218 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:08.219 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:08.219 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie 577c016b-6ffd-4f9f-8208-9ca7afd5e421 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:08.240 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:08.241 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:08.255 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:08.286 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:08.287 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:08.296 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:08.295+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:08.303 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:08.319 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:08.320 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:08.338 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:08.339 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:08.350 INFO:tasks.workunit.client.0.smithi165.stdout:/dev/nbd1 2024-09-21T23:39:08.358 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:08.358 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:08.358 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:08.358 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:08.358 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:08.359 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:08.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:08.378 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:08.378 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:08.397 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:08.397+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:08.416 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:08.417 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:08.417 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:08.417 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:08.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:08.438 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24098 2024-09-21T23:39:08.438 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24098 2024-09-21T23:39:08.438 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24098 -C rbd-nbd 2024-09-21T23:39:08.444 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:08.444 INFO:tasks.workunit.client.0.smithi165.stdout: 24098 ? 00:00:00 rbd-nbd 2024-09-21T23:39:08.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ ls /tmp/tmp.h0CUjCtN4b/mnt/ 2024-09-21T23:39:08.446 INFO:tasks.workunit.client.0.smithi165.stdout:lost+found 2024-09-21T23:39:08.446 INFO:tasks.workunit.client.0.smithi165.stdout:test 2024-09-21T23:39:08.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ dd if=/tmp/tmp.h0CUjCtN4b/mnt/test of=/dev/null bs=1M count=1 2024-09-21T23:39:08.453 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:39:08.454 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:39:08.454 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621566 s, 169 MB/s 2024-09-21T23:39:08.454 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo dd if=/tmp/tmp.h0CUjCtN4b/data of=/tmp/tmp.h0CUjCtN4b/mnt/test1 bs=1M count=1 oflag=direct 2024-09-21T23:39:08.454 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:08.454 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:08.455 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:08.455 INFO:tasks.workunit.client.0.smithi165.stderr:++ which dd 2024-09-21T23:39:08.456 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/dd 2024-09-21T23:39:08.456 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:08.456 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/dd if=/tmp/tmp.h0CUjCtN4b/data of=/tmp/tmp.h0CUjCtN4b/mnt/test1 bs=1M count=1 oflag=direct 2024-09-21T23:39:08.480 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records in 2024-09-21T23:39:08.480 INFO:tasks.workunit.client.0.smithi165.stderr:1+0 records out 2024-09-21T23:39:08.480 INFO:tasks.workunit.client.0.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169267 s, 61.9 MB/s 2024-09-21T23:39:08.481 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo umount /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:08.481 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:08.482 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:08.483 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:08.483 INFO:tasks.workunit.client.0.smithi165.stderr:++ which umount 2024-09-21T23:39:08.484 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/umount 2024-09-21T23:39:08.484 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:08.484 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/umount /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:08.498 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:08.497+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:08.499 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd1 24098 2024-09-21T23:39:08.499 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd1 2024-09-21T23:39:08.499 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=24098 2024-09-21T23:39:08.499 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:08.500 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:08.500 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:08.500 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:08.500 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:08.501 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:08.502 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:08.502 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:08.559 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:08.559 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:08.560 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:08.560 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:08.560 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:08.562 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:39:08.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:08.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:08.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^24098\b' 2024-09-21T23:39:08.595 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^24098\b' 2024-09-21T23:39:08.620 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:08.619+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:08.659 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:08.659 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:08.659 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *24098\b' 2024-09-21T23:39:08.660 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *24098\b' 2024-09-21T23:39:08.666 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:08.666 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:08.699 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:08.697+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.604559620000003, 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.701 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:08.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:39:08.706 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.590482072999997, 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:08.707 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:39:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:39:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.581390261999999, 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:39:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:39:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:39:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:39:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.559343165999998, 2024-09-21T23:39:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:39:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:39:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.455967833999999, 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.455838696000001, 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:39:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.424034278999997, 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:39:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:39:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:34 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.1812451670000002, 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023817232000000001, 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 34 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518869+0000", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518884+0000", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4742e-05 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518940+0000", 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5745999999999998e-05 2024-09-21T23:39:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518996+0000", 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5943999999999998e-05 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519161+0000", 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016499700000000001 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519412+0000", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025155899999999999 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519930+0000", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051799400000000003 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.537789+0000", 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017859223 2024-09-21T23:39:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542572+0000", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004782171 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542635+0000", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3083999999999994e-05 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542684+0000", 2024-09-21T23:39:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9135999999999998e-05 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:36 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.18105159, 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024223537999999999, 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519061+0000", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.283e-06 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519069+0000", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7190000000000001e-06 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519094+0000", 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4904000000000001e-05 2024-09-21T23:39:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519111+0000", 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7251000000000001e-05 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529102+0000", 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099906300000000003 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529230+0000", 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012755500000000001 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529483+0000", 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025336899999999998 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.539134+0000", 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096515980000000008 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542876+0000", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003741439 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543167+0000", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029152799999999999 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543284+0000", 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011626200000000001 2024-09-21T23:39:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:59 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.1319538009999999, 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018089152000000001, 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 59 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568160+0000", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9589999999999998e-06 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568169+0000", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0129999999999999e-06 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568192+0000", 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-05 2024-09-21T23:39:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568207+0000", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5804e-05 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568324+0000", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116344 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568338+0000", 2024-09-21T23:39:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4154e-05 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571148+0000", 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028103519999999999 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571230+0000", 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-05 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571406+0000", 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017574999999999999 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576139+0000", 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047326759999999999 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586043+0000", 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099038500000000005 2024-09-21T23:39:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586151+0000", 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108323 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586247+0000", 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6277e-05 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:92 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2125824~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0921304520000001, 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015425227, 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 92 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607982+0000", 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4900000000000001e-07 2024-09-21T23:39:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608111+0000", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012953800000000001 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608118+0000", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5180000000000001e-06 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608121+0000", 2024-09-21T23:39:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4989999999999999e-06 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614540+0000", 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064183850000000004 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614587+0000", 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7521000000000001e-05 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614665+0000", 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7497999999999996e-05 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617185+0000", 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025206109999999999 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623310+0000", 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006124694 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623355+0000", 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4842000000000002e-05 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623407+0000", 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1771999999999998e-05 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:93 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2256896~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0919846199999999, 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015447901, 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 93 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.41e-07 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608256+0000", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128291 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608264+0000", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7740000000000004e-06 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608269+0000", 2024-09-21T23:39:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2279999999999998e-06 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615063+0000", 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067938929999999996 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615159+0000", 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6428999999999994e-05 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615288+0000", 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012889800000000001 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617455+0000", 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021670169999999998 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623474+0000", 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060186069999999996 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623520+0000", 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6347999999999997e-05 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623575+0000", 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5074999999999997e-05 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:94 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2387968~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0918317989999999, 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017672687999999999, 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 94 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608281+0000", 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7800000000000001e-07 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608423+0000", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142819 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608433+0000", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.38e-06 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608440+0000", 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:39:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615922+0000", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074819919999999998 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615968+0000", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6622000000000002e-05 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616051+0000", 2024-09-21T23:39:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2434000000000004e-05 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617461+0000", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014100060000000001 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625814+0000", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083532390000000002 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625896+0000", 2024-09-21T23:39:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1805999999999996e-05 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625953+0000", 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6932000000000003e-05 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:95 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2519040~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.091660879, 2024-09-21T23:39:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017624719, 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608452+0000", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608524+0000", 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2565999999999996e-05 2024-09-21T23:39:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608532+0000", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2879999999999993e-06 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608539+0000", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616439+0000", 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007900041 2024-09-21T23:39:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616540+0000", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010035999999999999 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616676+0000", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0001362 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619529+0000", 2024-09-21T23:39:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028531250000000002 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625999+0000", 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064702960000000004 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626029+0000", 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9702000000000001e-05 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626076+0000", 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6656000000000002e-05 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:96 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2650112~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0915588899999999, 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017659136999999998, 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608620+0000", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6256999999999999e-05 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608634+0000", 2024-09-21T23:39:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608640+0000", 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1659999999999999e-06 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617495+0000", 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088556810000000007 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617544+0000", 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8162999999999999e-05 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617618+0000", 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4759999999999996e-05 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619535+0000", 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001916374 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626121+0000", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065866299999999996 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626166+0000", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4202999999999997e-05 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626212+0000", 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6434000000000003e-05 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:97 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2781184~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.091461695, 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017675217999999999, 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 97 2024-09-21T23:39:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608651+0000", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9299999999999998e-07 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608730+0000", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9598000000000006e-05 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608739+0000", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0119999999999994e-06 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608746+0000", 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2349999999999996e-06 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618001+0000", 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092555099999999998 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618092+0000", 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0552999999999999e-05 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618235+0000", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014355499999999999 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620378+0000", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021429750000000001 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626257+0000", 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0058784980000000002 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626283+0000", 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.601e-05 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626325+0000", 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2679000000000003e-05 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:98 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2912256~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0913561399999998, 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017700770000000001, 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 98 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608873+0000", 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117304 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608887+0000", 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3386e-05 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608894+0000", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618777+0000", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098831249999999995 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618825+0000", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8563000000000002e-05 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618902+0000", 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6130000000000005e-05 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621131+0000", 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022290230000000001 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626371+0000", 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052399329999999996 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626412+0000", 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1844999999999997e-05 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626456+0000", 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4013e-05 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:99 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3043328~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0912064770000001, 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017665733999999999, 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 99 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608906+0000", 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608993+0000", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6874999999999997e-05 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609004+0000", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0849999999999999e-05 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609010+0000", 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619684+0000", 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010673867 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619760+0000", 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6249999999999997e-05 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619880+0000", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011974200000000001 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622034+0000", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021542639999999999 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626501+0000", 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044669829999999999 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626528+0000", 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626571+0000", 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3123e-05 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:100 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3174400~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0910581750000001, 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01765369, 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609143+0000", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9022000000000002e-05 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609153+0000", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8160000000000005e-06 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609161+0000", 2024-09-21T23:39:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3420000000000008e-06 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620487+0000", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011325052 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620536+0000", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654999999999998e-05 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620615+0000", 2024-09-21T23:39:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8403000000000001e-05 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622168+0000", 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015539270000000001 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626619+0000", 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044508730000000002 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626662+0000", 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2175000000000002e-05 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626707+0000", 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5741999999999997e-05 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:101 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3305472~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.090937475, 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020329684000000001, 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609175+0000", 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2200000000000004e-07 2024-09-21T23:39:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609264+0000", 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8688000000000002e-05 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609273+0000", 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0380000000000005e-06 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609277+0000", 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.036e-06 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621290+0000", 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012013429000000001 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621385+0000", 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5260000000000006e-05 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621524+0000", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013853100000000001 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622385+0000", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00086160000000000002 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629426+0000", 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007040129 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629455+0000", 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9274000000000001e-05 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629504+0000", 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9076999999999998e-05 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:08.815 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:08.817 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:08.815+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:08.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:08.918 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:08.919+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:08.942 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:08.943 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:08.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:09.019 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:09.017+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:09.077 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:09.078 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:09.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:09.120 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:09.119+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:09.167 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n 577c016b-6ffd-4f9f-8208-9ca7afd5e421 ']' 2024-09-21T23:39:09.167 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd --show-cookie --cookie 'abc de' map rbd/testrbdnbd21555 2024-09-21T23:39:09.167 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:09.168 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:09.169 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:09.170 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:09.172 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:09.172 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:09.172 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie --cookie 'abc de' map rbd/testrbdnbd21555 2024-09-21T23:39:09.223 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:09.219+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:09.255 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.626289329, 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:39:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:39:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.622237247, 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:39:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:39:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:39:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:39:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.621423587, 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:39:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.577006931, 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:39:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.569626804000002, 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:39:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.521902781999998, 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:39:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.509908238999998, 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:39:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.82621812, 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:39:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:39:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:39:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.778544638, 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:39:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:39:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:39:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.755540433999997, 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:39:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:39:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:39:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.741702351999997, 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:39:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:39:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:25 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'135, mlcod=75'133)", 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.747600136, 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005751763, 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506953+0000", 2024-09-21T23:39:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506965+0000", 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-05 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506993+0000", 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8028999999999999e-05 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507006+0000", 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3417e-05 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507142+0000", 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013520200000000001 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507195+0000", 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3117e-05 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512636+0000", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054415619999999996 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512703+0000", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6863e-05 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:26 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'136, mlcod=75'133)", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7469105219999999, 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051681380000000001, 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507643+0000", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.773e-06 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507658+0000", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5136999999999999e-05 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507687+0000", 2024-09-21T23:39:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9306999999999999e-05 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507701+0000", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3376e-05 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508223+0000", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052228599999999995 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508266+0000", 2024-09-21T23:39:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3037000000000001e-05 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512707+0000", 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044412159999999996 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512809+0000", 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102006 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:28 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'138, mlcod=75'133)", 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7456649510000002, 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068616110000000001, 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508888+0000", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.753e-06 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508902+0000", 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3543999999999999e-05 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508928+0000", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5809e-05 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508944+0000", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6390000000000001e-05 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510106+0000", 2024-09-21T23:39:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001162185 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510155+0000", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8912000000000002e-05 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515650+0000", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054947529999999998 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515748+0000", 2024-09-21T23:39:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8264999999999995e-05 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:29 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'139, mlcod=75'133)", 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7450426930000003, 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063237359999999999, 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509511+0000", 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.747e-06 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509528+0000", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7047000000000001e-05 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509559+0000", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1463e-05 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509571+0000", 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2047e-05 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510923+0000", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001351882 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510973+0000", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9767e-05 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515755+0000", 2024-09-21T23:39:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00478269 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515833+0000", 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7093000000000002e-05 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:30 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'140, mlcod=75'133)", 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7443694839999999, 2024-09-21T23:39:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057255370000000002, 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510184+0000", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9360000000000002e-06 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510199+0000", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4564e-05 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510225+0000", 2024-09-21T23:39:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510250+0000", 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4323000000000001e-05 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511937+0000", 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016875010000000001 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511974+0000", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7425999999999997e-05 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515836+0000", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038614370000000001 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515908+0000", 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1717000000000005e-05 2024-09-21T23:39:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:47 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'149, mlcod=75'144)", 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7092802439999999, 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067526640000000002, 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545272+0000", 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6099999999999999e-07 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545279+0000", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4899999999999997e-06 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545293+0000", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3889e-05 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545301+0000", 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1980000000000001e-06 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545364+0000", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2890000000000003e-05 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545383+0000", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9074999999999999e-05 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.551967+0000", 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065840680000000002 2024-09-21T23:39:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552024+0000", 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7192999999999999e-05 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:48 2.5 e76/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 76'150, mlcod=75'147)", 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.707507251, 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050129090000000003, 2024-09-21T23:39:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547046+0000", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2470000000000001e-06 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547053+0000", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6029999999999999e-06 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547069+0000", 2024-09-21T23:39:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5863e-05 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547077+0000", 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4239999999999993e-06 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547146+0000", 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8106000000000001e-05 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547170+0000", 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4871999999999998e-05 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552029+0000", 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048586669999999997 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552057+0000", 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8127e-05 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:53 2.5 e76/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 76'156, mlcod=76'151)", 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.556370+0000", 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6981815309999999, 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004649355, 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556370+0000", 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556370+0000", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556371+0000", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.92e-07 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556377+0000", 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5989999999999998e-06 2024-09-21T23:39:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556391+0000", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3985000000000001e-05 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556398+0000", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1060000000000002e-06 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556445+0000", 2024-09-21T23:39:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6677e-05 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.556459+0000", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3368e-05 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.560934+0000", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044752940000000003 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.561019+0000", 2024-09-21T23:39:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5533999999999998e-05 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:55 2.5 e76/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 76'159, mlcod=76'153)", 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6967755090000001, 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065210290000000002, 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557781+0000", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.096e-06 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557790+0000", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1499999999999999e-06 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557794+0000", 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3749999999999996e-06 2024-09-21T23:39:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557832+0000", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8475e-05 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557845+0000", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2607e-05 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564252+0000", 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064069690000000002 2024-09-21T23:39:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564297+0000", 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5015000000000002e-05 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:09.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:09.350 INFO:tasks.workunit.client.0.smithi165.stderr:+ OUT='/dev/nbd1 abc de' 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:+ read DEV ANOTHER_COOKIE 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:09.351 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:09.352 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:09.347+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:09.437 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24172 2024-09-21T23:39:09.437 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24172 2024-09-21T23:39:09.437 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24172 -C rbd-nbd 2024-09-21T23:39:09.445 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:09.445 INFO:tasks.workunit.client.0.smithi165.stdout: 24172 ? 00:00:00 rbd-nbd 2024-09-21T23:39:09.445 INFO:tasks.workunit.client.0.smithi165.stderr:+ test 'abc de' = 'abc de' 2024-09-21T23:39:09.445 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd1 24172 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd1 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=24172 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:09.446 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:09.448 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:09.448 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:09.448 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd1 2024-09-21T23:39:09.450 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:09.447+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:09.474 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:09.474 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:09.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:09.551 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:09.547+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:09.558 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:09.558 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^24172\b' 2024-09-21T23:39:09.558 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^24172\b' 2024-09-21T23:39:09.614 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:09.614 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:09.620 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:09.622 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:09.622 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *24172\b' 2024-09-21T23:39:09.622 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *24172\b' 2024-09-21T23:39:09.628 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:09.628 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:09.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:09.673 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:09.673+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:09.754 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:09.755+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.350416377999998, 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:39:09.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:09.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.339157036000003, 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:09.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.317547902000001, 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:09.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.271452414999999, 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:39:09.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.259430328000001, 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:39:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:39:09.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.238133388000001, 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:39:09.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:39:09.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.227314925999998, 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:39:09.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:39:09.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:39:09.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.510204365, 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:09.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:09.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:39:09.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.510111535999997, 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:09.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.173287387000002, 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:39:09.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:39:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:39:09.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:39:09.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.152135014000002, 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:39:09.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:39:09.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:09.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:39:09.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.565904183000001, 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:39:09.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.430574655000001, 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:39:09.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:39:09.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:09.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.205525692000002, 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:39:09.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:39:09.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:33 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2373783060000001, 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019445529, 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2024-09-21T23:39:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515329+0000", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515340+0000", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0482999999999999e-05 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515363+0000", 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3193999999999998e-05 2024-09-21T23:39:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515379+0000", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5996000000000001e-05 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515512+0000", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132943 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515745+0000", 2024-09-21T23:39:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023335399999999999 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516012+0000", 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026738199999999998 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534312+0000", 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018299214000000001 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534516+0000", 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020432600000000001 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534653+0000", 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013709200000000001 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534772+0000", 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118942 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:35 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2360070829999996, 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018667975, 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516708+0000", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0077e-05 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516719+0000", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1683e-05 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516750+0000", 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0309000000000001e-05 2024-09-21T23:39:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516773+0000", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2762999999999999e-05 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523637+0000", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068647480000000004 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523818+0000", 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077600000000001 2024-09-21T23:39:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.524105+0000", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028720500000000001 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534471+0000", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010365740999999999 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535149+0000", 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067791800000000001 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535270+0000", 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120932 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535366+0000", 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5822999999999994e-05 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:34 2.2 e75/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 75'43, mlcod=58'40)", 2024-09-21T23:39:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2341673670000004, 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022102816000000001, 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518540+0000", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518554+0000", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4326e-05 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518580+0000", 2024-09-21T23:39:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5403000000000001e-05 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518596+0000", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518725+0000", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012880300000000001 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518759+0000", 2024-09-21T23:39:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4335999999999998e-05 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540576+0000", 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021817020999999999 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540640+0000", 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3917000000000005e-05 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.5215.0:33 2.6 e75/17 ondisk, result = 0)", 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2310507639999999, 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012683450000000001, 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521656+0000", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521668+0000", 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2092999999999999e-05 2024-09-21T23:39:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521695+0000", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6503999999999999e-05 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521714+0000", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9150000000000001e-05 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534274+0000", 2024-09-21T23:39:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012560135 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534295+0000", 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1118999999999999e-05 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534337+0000", 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2466000000000003e-05 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:36 2.2 e75/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 75'45, mlcod=58'40)", 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.223784899, 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0117643, 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528922+0000", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6129999999999998e-06 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528940+0000", 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7283e-05 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529003+0000", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3319999999999997e-05 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529024+0000", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.092e-05 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529224+0000", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019950499999999999 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529338+0000", 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114429 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540655+0000", 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011317117 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540684+0000", 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9113000000000001e-05 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:57 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.1943956370000004, 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013068644000000001, 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 57 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558312+0000", 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-21T23:39:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558326+0000", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3668e-05 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558359+0000", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3534000000000001e-05 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558376+0000", 2024-09-21T23:39:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6137000000000002e-05 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558480+0000", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104811 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558622+0000", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014147699999999999 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558881+0000", 2024-09-21T23:39:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025933899999999998 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564172+0000", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052903009999999999 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571263+0000", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070919110000000002 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571332+0000", 2024-09-21T23:39:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8733999999999996e-05 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571378+0000", 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5629e-05 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:09.862 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:09.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:09.864 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:09.861+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:09.964 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:09.965+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:10.034 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:10.034 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:10.034 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:10.034 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:10.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:10.065 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:10.063+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:10.129 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:10.130 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd snap ls rbd/testrbdnbd21555 2024-09-21T23:39:10.130 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '$2 == "snap" {print $1}' 2024-09-21T23:39:10.166 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:10.163+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:10.197 INFO:tasks.workunit.client.0.smithi165.stderr:+ SNAPID=3 2024-09-21T23:39:10.198 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd --show-cookie map --snap-id 3 rbd/testrbdnbd21555 2024-09-21T23:39:10.198 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:10.198 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:10.198 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:10.198 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:10.199 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:10.199 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:10.199 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --show-cookie map --snap-id 3 rbd/testrbdnbd21555 2024-09-21T23:39:10.217 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:10.217 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:10.218 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:10.218 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:10.218 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:10.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:10.281 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:10.281+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:10.337 INFO:tasks.workunit.client.0.smithi165.stderr:+ OUT='/dev/nbd1 4f85eb7a-0b3e-4433-89e4-c9f289c52803' 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:+ read DEV COOKIE 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:10.338 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.050681994000001, 2024-09-21T23:39:10.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:39:10.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:39:10.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:39:10.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:39:10.362 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:10.362 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.049818420999998, 2024-09-21T23:39:10.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:10.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:39:10.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:39:10.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:39:10.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.049155910000003, 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.371 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:39:10.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:39:10.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.048481840999997, 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.376 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:39:10.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.047755260000002, 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:39:10.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:39:10.385 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.047029639000002, 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.046292667000003, 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:10.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:39:10.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:39:10.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.394 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.405662704999997, 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:39:10.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.391889128999999, 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:10.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.329710726000002, 2024-09-21T23:39:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:39:10.404 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:39:10.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:31 2.0 e75/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 75'89, mlcod=72'87)", 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8373146250000003, 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065639640000000003, 2024-09-21T23:39:10.409 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514499+0000", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.877e-06 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514510+0000", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1663e-05 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514540+0000", 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0096000000000001e-05 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514560+0000", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9317000000000002e-05 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514684+0000", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012411200000000001 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514721+0000", 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7707000000000003e-05 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.520975+0000", 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062540119999999998 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521061+0000", 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5179999999999994e-05 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:41 2.0 e75/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 75'91, mlcod=72'88)", 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8165309990000003, 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089977249999999998, 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535283+0000", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535304+0000", 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0619999999999999e-05 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535351+0000", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7719999999999997e-05 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535373+0000", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1552999999999998e-05 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535531+0000", 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000157885 2024-09-21T23:39:10.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535593+0000", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2399999999999999e-05 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544205+0000", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086119019999999994 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544278+0000", 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2974999999999998e-05 2024-09-21T23:39:10.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:43 2.3 e75/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 75'98, mlcod=64'95)", 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8152731949999996, 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010489332000000001, 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536540+0000", 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536561+0000", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0933000000000001e-05 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536599+0000", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7963000000000002e-05 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536618+0000", 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9040000000000001e-05 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536799+0000", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018089199999999999 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536869+0000", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0436000000000001e-05 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.546957+0000", 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010087631999999999 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547027+0000", 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0363000000000004e-05 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:92 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'108, mlcod=76'100)", 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7386762529999995, 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083907579999999999, 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.422 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613136+0000", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4499999999999996e-07 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613361+0000", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022507399999999999 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613378+0000", 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6731999999999998e-05 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613391+0000", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3047000000000001e-05 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615189+0000", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001797931 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5012000000000003e-05 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621392+0000", 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061686850000000001 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621526+0000", 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013353200000000001 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:93 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'109, mlcod=76'100)", 2024-09-21T23:39:10.426 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7380267289999995, 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077928269999999996, 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0699999999999996e-07 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613964+0000", 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017884699999999999 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613979+0000", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243000000000001e-05 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613987+0000", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7900000000000005e-06 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616257+0000", 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022696130000000002 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616287+0000", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0450000000000002e-05 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621532+0000", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052442909999999999 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621577+0000", 2024-09-21T23:39:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5686000000000003e-05 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:94 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'110, mlcod=76'100)", 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7372577429999998, 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086000920000000002, 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614657+0000", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102179 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614668+0000", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614675+0000", 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-21T23:39:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617112+0000", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024372949999999999 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617146+0000", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4162999999999998e-05 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623080+0000", 2024-09-21T23:39:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059341530000000002 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623154+0000", 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3212000000000003e-05 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:95 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'111, mlcod=76'100)", 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7365886560000003, 2024-09-21T23:39:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079773919999999998, 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6599999999999994e-07 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615366+0000", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000142585 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615383+0000", 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7244999999999998e-05 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615394+0000", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0715e-05 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618154+0000", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027597450000000001 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618188+0000", 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4217000000000001e-05 2024-09-21T23:39:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623158+0000", 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049695989999999999 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623200+0000", 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2320000000000001e-05 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:96 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'112, mlcod=76'102)", 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7357012909999998, 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071258629999999996, 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616179+0000", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8514000000000002e-05 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616260+0000", 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1724999999999994e-05 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616274+0000", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3884e-05 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616283+0000", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2289999999999992e-06 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618980+0000", 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026965980000000001 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618993+0000", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2835e-05 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623203+0000", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042096950000000003 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623236+0000", 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3383000000000002e-05 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:97 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'113, mlcod=76'102)", 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7350741310000002, 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065371459999999998, 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616738+0000", 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616847+0000", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010869700000000001 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616860+0000", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145e-05 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616867+0000", 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2019999999999998e-06 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619479+0000", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002611642 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619511+0000", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2246000000000003e-05 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623237+0000", 2024-09-21T23:39:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037264609999999999 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623274+0000", 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6925999999999998e-05 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:101 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'117, mlcod=76'106)", 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.731720514, 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066386559999999997, 2024-09-21T23:39:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620092+0000", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7199999999999997e-07 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620282+0000", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019049 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620317+0000", 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4402e-05 2024-09-21T23:39:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620331+0000", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4338000000000001e-05 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622069+0000", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001738126 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622100+0000", 2024-09-21T23:39:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1272999999999997e-05 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626643+0000", 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004543025 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626729+0000", 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6130000000000004e-05 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:10.453 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:10.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:10.454 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24256 2024-09-21T23:39:10.454 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24256 2024-09-21T23:39:10.454 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24256 -C rbd-nbd 2024-09-21T23:39:10.455 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:10.455 INFO:tasks.workunit.client.0.smithi165.stdout: 24256 ? 00:00:00 rbd-nbd 2024-09-21T23:39:10.456 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:10.456 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:10.453+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device detach rbd/testrbdnbd21555 --snap-id 3 --device-type nbd 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:10.457 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device detach rbd/testrbdnbd21555 --snap-id 3 --device-type nbd 2024-09-21T23:39:10.555 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:10.551+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:10.567 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false get_pid rbd 2024-09-21T23:39:10.568 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:10.568 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:10.568 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:10.568 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:10.568 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:10.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:39:10.589 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:10.589 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:10.590 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:10.590 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:10.604 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:10.636 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID= 2024-09-21T23:39:10.636 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n '' 2024-09-21T23:39:10.636 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 1 2024-09-21T23:39:10.636 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false _sudo rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:10.637 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:10.638 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:10.639 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:10.639 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --snap-id 3 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:10.642 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:10.660 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:10.675 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: could not validate attach request 2024-09-21T23:39:10.675 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: mismatching the image and the device may lead to data corruption 2024-09-21T23:39:10.675 INFO:tasks.workunit.client.0.smithi165.stderr:rbd: must specify --cookie or --force to proceed 2024-09-21T23:39:10.676 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:10.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:10.678 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:10.678 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n 4f85eb7a-0b3e-4433-89e4-c9f289c52803 ']' 2024-09-21T23:39:10.678 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device attach --device /dev/nbd1 --cookie 4f85eb7a-0b3e-4433-89e4-c9f289c52803 --snap-id 3 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:10.678 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:10.678 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:10.679 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:10.679 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:10.680 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:10.680 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:10.680 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device attach --device /dev/nbd1 --cookie 4f85eb7a-0b3e-4433-89e4-c9f289c52803 --snap-id 3 rbd/testrbdnbd21555 --device-type nbd 2024-09-21T23:39:10.692 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:10.691+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:10.695 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:10.695 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:10.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:10.715 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:10.734 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:10.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:10.756 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:10.755+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:10.775 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:10.776 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:10.790 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:10.813 INFO:tasks.workunit.client.0.smithi165.stdout:/dev/nbd1 2024-09-21T23:39:10.820 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:10.820 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:10.820 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:10.820 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:10.821 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:10.876 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:10.875+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:10.902 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24320 2024-09-21T23:39:10.902 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24320 2024-09-21T23:39:10.902 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24320 -C rbd-nbd 2024-09-21T23:39:10.908 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:10.908 INFO:tasks.workunit.client.0.smithi165.stdout: 24320 ? 00:00:00 rbd-nbd 2024-09-21T23:39:10.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device detach /dev/nbd1 --device-type nbd 2024-09-21T23:39:10.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:10.909 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:10.909 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:10.910 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:10.911 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:10.911 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:10.911 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device detach /dev/nbd1 --device-type nbd 2024-09-21T23:39:10.958 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:10.955+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:10.958 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:10.959 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:10.973 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:11.005 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false get_pid rbd 2024-09-21T23:39:11.006 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:11.006 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:11.006 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:11.006 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:11.006 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd1'\'']/id' 2024-09-21T23:39:11.060 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:11.059+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID= 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n '' 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 1 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:11.076 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd config image set rbd/testrbdnbd21555 rbd_discard_granularity_bytes 4096 2024-09-21T23:39:11.139 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.139 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.156 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:11.174 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd feature enable rbd/testrbdnbd21555 journaling 2024-09-21T23:39:11.200 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:11.199+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:11.260 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:11.261+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:11.270 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:11.270 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:11.270 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:11.271 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.271 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:11.273 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:11.273 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:11.273 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:11.323 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.323 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.337 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:11.361 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:11.361+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:11.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd2 2024-09-21T23:39:11.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:11.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:11.444 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:11.444 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:11.445 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd2'\'']/id' 2024-09-21T23:39:11.462 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:11.459+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:11.464 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.464 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:11.544 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24419 2024-09-21T23:39:11.544 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24419 2024-09-21T23:39:11.544 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24419 -C rbd-nbd 2024-09-21T23:39:11.550 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:11.550 INFO:tasks.workunit.client.0.smithi165.stdout: 24419 ? 00:00:00 rbd-nbd 2024-09-21T23:39:11.550 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 0 --length 3584 /dev/nbd2 2024-09-21T23:39:11.550 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.550 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.551 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.552 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.553 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.553 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.554 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 3584 /dev/nbd2 2024-09-21T23:39:11.568 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:11.567+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:11.646 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.647 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.660 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:11.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:39:11.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:39:11.682 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:39:11.682 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.695571 2024-09-21T23:39:11.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.6955708135543727 2024-09-21T23:39:11.690 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:39:11.704 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:11.703+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:11.764 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:11.765+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:11.787 INFO:tasks.workunit.client.0.smithi165.stderr:blkdiscard: /dev/nbd2 contains existing file system (ext2). 2024-09-21T23:39:11.789 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-21T23:39:11.789 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.790 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.791 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.792 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.794 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-21T23:39:11.823 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 0 --length 4608 /dev/nbd2 2024-09-21T23:39:11.824 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.824 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.826 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.826 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.828 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4608 /dev/nbd2 2024-09-21T23:39:11.829 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.829 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:11.845 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 512 --length 7168 /dev/nbd2 2024-09-21T23:39:11.845 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.845 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.847 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.847 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.849 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.849 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.849 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 7168 /dev/nbd2 2024-09-21T23:39:11.860 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 512 --length 7680 /dev/nbd2 2024-09-21T23:39:11.860 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.860 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.861 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.861 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.863 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 7680 /dev/nbd2 2024-09-21T23:39:11.878 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 512 --length 8192 /dev/nbd2 2024-09-21T23:39:11.878 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:11.879 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:11.879 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:11.880 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:11.881 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:11.881 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:11.882 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 512 --length 8192 /dev/nbd2 2024-09-21T23:39:11.888 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:11.891+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:11.897 INFO:tasks.workunit.client.0.smithi165.stderr:+ tries=0 2024-09-21T23:39:11.898 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21555 2024-09-21T23:39:11.898 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-21T23:39:11.961 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:11.961 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:11.966 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:11.967+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:11.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:11.984 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:11.982+0000 7f507e635640 -1 JournalMetadata: 0x56378829c560 failed to locate client: INSPECT 2024-09-21T23:39:11.992 INFO:tasks.workunit.client.0.smithi165.stderr:+ queue_length=4 2024-09-21T23:39:11.992 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 0 -lt 10 ']' 2024-09-21T23:39:11.992 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 4 -gt 0 ']' 2024-09-21T23:39:11.992 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21555 --verbose 2024-09-21T23:39:12.060 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=1 2024-09-21T23:39:12.060 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.816555+0000" 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=2 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.839500+0000" 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:12.061 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=3 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.872443+0000" 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=4 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:12.062 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.891682+0000" 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout:Summary: 2024-09-21T23:39:12.063 INFO:tasks.workunit.client.0.smithi165.stdout: 4 entries inspected, 0 errors 2024-09-21T23:39:12.067 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:12.066+0000 7f7ada7fc640 -1 JournalMetadata: 0x5582d6e28980 failed to locate client: INSPECT 2024-09-21T23:39:12.067 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:12.065+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:12.071 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 1 2024-09-21T23:39:12.111 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:12.111 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:12.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:12.168 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:12.171+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:12.268 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:12.269+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:12.292 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:12.292 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:12.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:12.393 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:12.393+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:12.470 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:12.469+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:12.471 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:12.471 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:12.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:12.586 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:12.585+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:12.653 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:12.653 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:12.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:12.673 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:12.675+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:12.688 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 0.695571 (8455842) 2024-09-21T23:39:12.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:12.774 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:12.775+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:12.810 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:12.810 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:12.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:12.875 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:12.875+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:12.891 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:12.891 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:12.908 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:12.908 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:12.924 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:12.925 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:12.945 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:12.949 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.547183695999998, 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:39:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:39:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:39:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.535924354000002, 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:39:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:39:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.51431522, 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:39:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:39:12.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.468219732999998, 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:12.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:39:12.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.456197646, 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:39:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:39:12.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.434900706000001, 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:39:12.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:39:12.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.424082243999997, 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:39:12.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:39:12.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:12.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 44.706971682999999, 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:12.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:39:12.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:12.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 44.706878854000003, 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:12.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:39:13.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.370054705000001, 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:39:13.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:13.010 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.348902332000002, 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:39:13.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:39:13.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.762671501, 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:39:13.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.627341973, 2024-09-21T23:39:13.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:39:13.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.402293010000001, 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:39:13.028 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:39:13.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:33 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.4341456240000001, 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019445529, 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515329+0000", 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515340+0000", 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0482999999999999e-05 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515363+0000", 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3193999999999998e-05 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515379+0000", 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5996000000000001e-05 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515512+0000", 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132943 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515745+0000", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023335399999999999 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516012+0000", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026738199999999998 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534312+0000", 2024-09-21T23:39:13.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018299214000000001 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534516+0000", 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020432600000000001 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534653+0000", 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013709200000000001 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534772+0000", 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118942 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:35 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.4327744009999996, 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018667975, 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516708+0000", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0077e-05 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516719+0000", 2024-09-21T23:39:13.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1683e-05 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516750+0000", 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0309000000000001e-05 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516773+0000", 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2762999999999999e-05 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523637+0000", 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068647480000000004 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523818+0000", 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077600000000001 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.524105+0000", 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028720500000000001 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534471+0000", 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010365740999999999 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535149+0000", 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067791800000000001 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535270+0000", 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120932 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535366+0000", 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5822999999999994e-05 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:34 2.2 e75/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 75'43, mlcod=58'40)", 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.4309346850000004, 2024-09-21T23:39:13.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022102816000000001, 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518540+0000", 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518554+0000", 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4326e-05 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518580+0000", 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5403000000000001e-05 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518596+0000", 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518725+0000", 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012880300000000001 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518759+0000", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4335999999999998e-05 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540576+0000", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021817020999999999 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540640+0000", 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3917000000000005e-05 2024-09-21T23:39:13.052 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.5215.0:33 2.6 e75/17 ondisk, result = 0)", 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.4278180819999999, 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012683450000000001, 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521656+0000", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521668+0000", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2092999999999999e-05 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521695+0000", 2024-09-21T23:39:13.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6503999999999999e-05 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521714+0000", 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9150000000000001e-05 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534274+0000", 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012560135 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534295+0000", 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1118999999999999e-05 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534337+0000", 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2466000000000003e-05 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.057 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:36 2.2 e75/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 75'45, mlcod=58'40)", 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.420552217, 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0117643, 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528922+0000", 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6129999999999998e-06 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528940+0000", 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7283e-05 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529003+0000", 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3319999999999997e-05 2024-09-21T23:39:13.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529024+0000", 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.092e-05 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529224+0000", 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019950499999999999 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529338+0000", 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114429 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540655+0000", 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011317117 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540684+0000", 2024-09-21T23:39:13.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9113000000000001e-05 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:57 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:13.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3911629550000004, 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013068644000000001, 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 57 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558312+0000", 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558326+0000", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3668e-05 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558359+0000", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3534000000000001e-05 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558376+0000", 2024-09-21T23:39:13.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6137000000000002e-05 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558480+0000", 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104811 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558622+0000", 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014147699999999999 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:13.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558881+0000", 2024-09-21T23:39:13.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025933899999999998 2024-09-21T23:39:13.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:13.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564172+0000", 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052903009999999999 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571263+0000", 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070919110000000002 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571332+0000", 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8733999999999996e-05 2024-09-21T23:39:13.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571378+0000", 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5629e-05 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.071 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:13.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:13.073 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:13.074 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:13.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:13.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:13.075 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:13.073+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:13.075 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21555 2024-09-21T23:39:13.075 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-21T23:39:13.174 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:13.173+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:13.249 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:13.250 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:13.250 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:13.250 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:13.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:13.305 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:13.305+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:13.376 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:13.373+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:13.422 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:13.422 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:13.423 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:13.423 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:13.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:13.481 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:13.481+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:13.577 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:13.579+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.297954769, 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:39:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.297091195999997, 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:39:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:39:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.296428685000002, 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.295754616000004, 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:39:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.295028035000001, 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:39:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.294302414000001, 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:39:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:39:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.293565442000002, 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:39:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:716 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.652935479999996, 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065233940000000001, 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946149+0000", 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.946196+0000", 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7862999999999999e-05 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947252+0000", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010554920000000001 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947280+0000", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7866000000000001e-05 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947294+0000", 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.379e-05 2024-09-21T23:39:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947411+0000", 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000117089 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.947449+0000", 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8720000000000002e-05 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952596+0000", 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051468939999999999 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.952672+0000", 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5679999999999994e-05 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.639161903999998, 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.576983501000001, 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:39:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:39:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:39:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:31 2.0 e75/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 75'89, mlcod=72'87)", 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0845874000000002, 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065639640000000003, 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514499+0000", 2024-09-21T23:39:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.877e-06 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514510+0000", 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1663e-05 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514540+0000", 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0096000000000001e-05 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514560+0000", 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9317000000000002e-05 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514684+0000", 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012411200000000001 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514721+0000", 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7707000000000003e-05 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.520975+0000", 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062540119999999998 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521061+0000", 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5179999999999994e-05 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:41 2.0 e75/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 75'91, mlcod=72'88)", 2024-09-21T23:39:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0638037740000001, 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089977249999999998, 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535283+0000", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535304+0000", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0619999999999999e-05 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535351+0000", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7719999999999997e-05 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535373+0000", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1552999999999998e-05 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535531+0000", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000157885 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535593+0000", 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2399999999999999e-05 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544205+0000", 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086119019999999994 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544278+0000", 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2974999999999998e-05 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:43 2.3 e75/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 75'98, mlcod=64'95)", 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0625459700000004, 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010489332000000001, 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536540+0000", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536561+0000", 2024-09-21T23:39:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0933000000000001e-05 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536599+0000", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7963000000000002e-05 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536618+0000", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9040000000000001e-05 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536799+0000", 2024-09-21T23:39:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018089199999999999 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536869+0000", 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0436000000000001e-05 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.546957+0000", 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010087631999999999 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547027+0000", 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0363000000000004e-05 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:92 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'108, mlcod=76'100)", 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9859490280000003, 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083907579999999999, 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613136+0000", 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4499999999999996e-07 2024-09-21T23:39:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613361+0000", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022507399999999999 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613378+0000", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6731999999999998e-05 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613391+0000", 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3047000000000001e-05 2024-09-21T23:39:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615189+0000", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001797931 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5012000000000003e-05 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621392+0000", 2024-09-21T23:39:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061686850000000001 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621526+0000", 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013353200000000001 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:93 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'109, mlcod=76'100)", 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9852995040000003, 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077928269999999996, 2024-09-21T23:39:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0699999999999996e-07 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613964+0000", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017884699999999999 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613979+0000", 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243000000000001e-05 2024-09-21T23:39:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613987+0000", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7900000000000005e-06 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616257+0000", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022696130000000002 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616287+0000", 2024-09-21T23:39:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0450000000000002e-05 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621532+0000", 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052442909999999999 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621577+0000", 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5686000000000003e-05 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:94 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'110, mlcod=76'100)", 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9845305179999997, 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086000920000000002, 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614657+0000", 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102179 2024-09-21T23:39:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614668+0000", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614675+0000", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617112+0000", 2024-09-21T23:39:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024372949999999999 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617146+0000", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4162999999999998e-05 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623080+0000", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059341530000000002 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623154+0000", 2024-09-21T23:39:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3212000000000003e-05 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:95 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'111, mlcod=76'100)", 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9838614310000002, 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079773919999999998, 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6599999999999994e-07 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615366+0000", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000142585 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615383+0000", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7244999999999998e-05 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615394+0000", 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0715e-05 2024-09-21T23:39:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618154+0000", 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027597450000000001 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618188+0000", 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4217000000000001e-05 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623158+0000", 2024-09-21T23:39:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049695989999999999 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623200+0000", 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2320000000000001e-05 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:96 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'112, mlcod=76'102)", 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9829740659999997, 2024-09-21T23:39:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071258629999999996, 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616179+0000", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8514000000000002e-05 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616260+0000", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1724999999999994e-05 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616274+0000", 2024-09-21T23:39:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3884e-05 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616283+0000", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2289999999999992e-06 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618980+0000", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026965980000000001 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618993+0000", 2024-09-21T23:39:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2835e-05 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623203+0000", 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042096950000000003 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623236+0000", 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3383000000000002e-05 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:97 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'113, mlcod=76'102)", 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9823469060000001, 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065371459999999998, 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616738+0000", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616847+0000", 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010869700000000001 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616860+0000", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145e-05 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616867+0000", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2019999999999998e-06 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619479+0000", 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002611642 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619511+0000", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2246000000000003e-05 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623237+0000", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037264609999999999 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623274+0000", 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6925999999999998e-05 2024-09-21T23:39:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:101 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'117, mlcod=76'106)", 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9789932889999999, 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066386559999999997, 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620092+0000", 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7199999999999997e-07 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620282+0000", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019049 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620317+0000", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4402e-05 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620331+0000", 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4338000000000001e-05 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622069+0000", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001738126 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622100+0000", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1272999999999997e-05 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626643+0000", 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004543025 2024-09-21T23:39:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626729+0000", 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6130000000000004e-05 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:13.695 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:13.697 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:13.697 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:13.697+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:13.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:39:13.723 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:13.722+0000 7f219effd640 -1 JournalMetadata: 0x556276cf3780 failed to locate client: INSPECT 2024-09-21T23:39:13.732 INFO:tasks.workunit.client.0.smithi165.stderr:+ queue_length=4 2024-09-21T23:39:13.732 INFO:tasks.workunit.client.0.smithi165.stderr:+ tries=1 2024-09-21T23:39:13.732 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1 -lt 10 ']' 2024-09-21T23:39:13.732 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 4 -gt 0 ']' 2024-09-21T23:39:13.733 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21555 --verbose 2024-09-21T23:39:13.797 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:13.799+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=1 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:13.811 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.816555+0000" 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=2 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.839500+0000" 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=3 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:13.812 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.872443+0000" 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=4 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:13.813 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.891682+0000" 2024-09-21T23:39:13.814 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:13.814 INFO:tasks.workunit.client.0.smithi165.stdout:Summary: 2024-09-21T23:39:13.814 INFO:tasks.workunit.client.0.smithi165.stdout: 4 entries inspected, 0 errors 2024-09-21T23:39:13.815 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:13.814+0000 7fde5affd640 -1 JournalMetadata: 0x556e1180b990 failed to locate client: INSPECT 2024-09-21T23:39:13.821 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 1 2024-09-21T23:39:13.830 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:13.830 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:13.830 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:13.830 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:13.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:13.936 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:13.939+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:13.999 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:13.997+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:14.016 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:14.017 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:14.033 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:14.100 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:14.097+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:14.201 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:14.201 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:14.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.970015539, 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:39:14.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:39:14.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:39:14.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:39:14.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.622710296000001, 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.209 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:39:14.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:14.214 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.622630862999998, 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:14.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:39:14.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:39:14.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.622599860000001, 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.218 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:39:14.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.622555302000002, 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.222 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:39:14.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.622520496999996, 2024-09-21T23:39:14.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:14.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.230 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598907273999998, 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:39:14.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:39:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.236 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598872358000001, 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:39:14.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.242 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598840609, 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.248 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598800947999997, 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:39:14.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598769709000003, 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:39:14.254 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:39:14.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:39:14.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.598315331999999, 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:39:14.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:39:14.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.597941147999997, 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.266 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:39:14.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:39:14.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.271 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.595683739000002, 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:39:14.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:39:14.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.591340054, 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:39:14.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:39:14.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:14.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.585176431000001, 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:14.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:39:14.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:39:14.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.581946535999997, 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:39:14.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:39:14.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:39:14.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.581560279999998, 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:14.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:39:14.302 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.571798899999997, 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:39:14.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:39:14.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.571569642999997, 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:14.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:39:14.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:39:14.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:14.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.315 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.315 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:14.316 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:14.201+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:14.316 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:14.317 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:14.313+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:14.416 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:14.417+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:14.443 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:14.443 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:14.443 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:14.443 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:14.458 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:14.517 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:14.519+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:14.581 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:14.582 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:14.595 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:14.636 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:14.639+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:14.701 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:39:14.719 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:14.719+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:14.762 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:14.762 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.399995380999997, 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:39:14.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.388331342000001, 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.767 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:39:14.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.771 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.376765112999998, 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:39:14.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:39:14.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.890827577000003, 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:39:14.776 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:39:14.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.866917460000003, 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:39:14.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.858426399000003, 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:39:14.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:39:14.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.839290814999998, 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.829886494999997, 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:39:14.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:39:14.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:39:14.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.184451369000001, 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:39:14.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.152796985999998, 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:39:14.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:39:14.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:39:14.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.149093911999998, 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.807 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:39:14.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:39:14.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:39:14.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.593490482999997, 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:14.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:39:14.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.134943231999998, 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:39:14.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:39:14.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.123579974000002, 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:39:14.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:39:14.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.112185017000002, 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.102576278000001, 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:39:14.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:39:14.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:32 2.7 e75/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 75'154, mlcod=75'151)", 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2472202459999995, 2024-09-21T23:39:14.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010506980000000001, 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516298+0000", 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0040000000000002e-06 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516317+0000", 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9060999999999999e-05 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516356+0000", 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8831999999999997e-05 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516380+0000", 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4139999999999999e-05 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516532+0000", 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015149900000000001 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516573+0000", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1007999999999998e-05 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526749+0000", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010175797 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526803+0000", 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639000000000003e-05 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:60 2.7 e76/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 76'156, mlcod=75'152)", 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.194194843, 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071394079999999999, 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:14.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999996e-07 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569332+0000", 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0203e-05 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569361+0000", 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8127999999999999e-05 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569390+0000", 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9263000000000001e-05 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569468+0000", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8051000000000002e-05 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569493+0000", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5313e-05 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576410+0000", 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069164969999999997 2024-09-21T23:39:14.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576461+0000", 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1348999999999999e-05 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:64 2.7 e76/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 76'158, mlcod=75'152)", 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.1865043259999997, 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042077199999999999, 2024-09-21T23:39:14.842 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577014+0000", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.612e-06 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577027+0000", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3570999999999999e-05 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577054+0000", 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6713999999999999e-05 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577070+0000", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577138+0000", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8003999999999995e-05 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577157+0000", 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9587000000000001e-05 2024-09-21T23:39:14.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581148+0000", 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039903500000000001 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581220+0000", 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2174000000000004e-05 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5254.0:37 2.7 e77/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 77'184, mlcod=77'180)", 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.2871872130000002, 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029648180000000001, 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476330+0000", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476454+0000", 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012381499999999999 2024-09-21T23:39:14.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476468+0000", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3737000000000001e-05 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476477+0000", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477554+0000", 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010771789999999999 2024-09-21T23:39:14.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477568+0000", 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577000000000001e-05 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479264+0000", 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001696637 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:14.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479294+0000", 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9686999999999999e-05 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:14.851 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:14.852 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21555 2024-09-21T23:39:14.852 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-21T23:39:14.853 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:14.853 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:14.855+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:14.929 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:14.926+0000 7f0725ffb640 -1 JournalMetadata: 0x55578114f5d0 failed to locate client: INSPECT 2024-09-21T23:39:14.935 INFO:tasks.workunit.client.0.smithi165.stderr:+ queue_length=4 2024-09-21T23:39:14.935 INFO:tasks.workunit.client.0.smithi165.stderr:+ tries=2 2024-09-21T23:39:14.935 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 2 -lt 10 ']' 2024-09-21T23:39:14.935 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 4 -gt 0 ']' 2024-09-21T23:39:14.935 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21555 --verbose 2024-09-21T23:39:14.954 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:14.953+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:15.016 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=1 2024-09-21T23:39:15.016 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.816555+0000" 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=2 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:15.017 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.839500+0000" 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=3 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.872443+0000" 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:15.018 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=4 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.891682+0000" 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout:Summary: 2024-09-21T23:39:15.019 INFO:tasks.workunit.client.0.smithi165.stdout: 4 entries inspected, 0 errors 2024-09-21T23:39:15.023 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:15.022+0000 7fb11bfff640 -1 JournalMetadata: 0x55ad89d4a3b0 failed to locate client: INSPECT 2024-09-21T23:39:15.028 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:15.028 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:15.028 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:15.028 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:15.030 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 1 2024-09-21T23:39:15.042 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:15.054 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:15.053+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:15.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:15.155 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:15.155+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:15.159 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:15.159 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:15.159 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:15.160 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:15.160 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:15.173 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:15.231 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:15.231 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:15.247 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:15.247 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:15.260 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:15.263+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:15.264 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:15.264 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:15.282 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:15.282 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:15.301 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:15.301 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:15.320 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:15.320 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:15.336 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:15.336 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:15.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:15.393 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:15.395+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:15.458 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:15.459+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:15.468 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:15.468 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:15.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:15.559 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:15.559+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:15.605 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:15.605 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:15.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:15.669 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:15.669+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:15.745 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:15.745 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:15.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:15.761 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:15.763+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:15.861 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:15.861+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:15.930 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:15.930 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:15.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:15.962 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:15.963+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:16.032 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21555 2024-09-21T23:39:16.032 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-21T23:39:16.064 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:16.063+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:16.113 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.113 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.127 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:16.126+0000 7f660dffd640 -1 JournalMetadata: 0x559c31c7ed60 failed to locate client: INSPECT 2024-09-21T23:39:16.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:16.134 INFO:tasks.workunit.client.0.smithi165.stderr:+ queue_length=4 2024-09-21T23:39:16.134 INFO:tasks.workunit.client.0.smithi165.stderr:+ tries=3 2024-09-21T23:39:16.134 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 3 -lt 10 ']' 2024-09-21T23:39:16.134 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 4 -gt 0 ']' 2024-09-21T23:39:16.134 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd journal inspect --pool rbd --image testrbdnbd21555 --verbose 2024-09-21T23:39:16.165 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:16.167+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:16.208 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=1 2024-09-21T23:39:16.208 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:16.208 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:16.208 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:16.208 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.816555+0000" 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=2 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 0, 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.839500+0000" 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=3 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:16.209 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.872443+0000" 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout:Entry: tag_id=1, commit_tid=4 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout:{ 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "event_type": "AioDiscard", 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "offset": 4096, 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "length": 4096, 2024-09-21T23:39:16.210 INFO:tasks.workunit.client.0.smithi165.stdout: "discard_granularity_bytes": 4096, 2024-09-21T23:39:16.211 INFO:tasks.workunit.client.0.smithi165.stdout: "timestamp": "2024-09-21T23:39:11.891682+0000" 2024-09-21T23:39:16.211 INFO:tasks.workunit.client.0.smithi165.stdout:} 2024-09-21T23:39:16.211 INFO:tasks.workunit.client.0.smithi165.stdout:Summary: 2024-09-21T23:39:16.211 INFO:tasks.workunit.client.0.smithi165.stdout: 4 entries inspected, 0 errors 2024-09-21T23:39:16.211 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:16.210+0000 7fa301e08640 -1 JournalMetadata: 0x55ca80e37750 failed to locate client: INSPECT 2024-09-21T23:39:16.216 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 1 2024-09-21T23:39:16.249 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.249 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:16.265 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:16.267+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:16.365 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:16.365+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:16.384 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.384 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:16.485 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:16.485+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:16.563 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.563 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.567 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:16.566+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:16.577 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:16.669 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:16.670+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:16.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:39:16.744 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.744 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.759 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:16.805 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:16.806+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:16.870 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:16.870+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:16.890 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:16.890 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:16.904 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:16.971 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:16.969+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:17.072 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:17.070+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:17.073 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:17.073 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:17.087 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:17.173 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:17.174+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:17.219 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd journal inspect --pool rbd --image testrbdnbd21555 2024-09-21T23:39:17.219 INFO:tasks.workunit.client.0.smithi165.stderr:++ awk '/entries inspected/ {print $1}' 2024-09-21T23:39:17.245 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:17.246 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:17.260 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:17.301 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:17.302+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:17.303 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:17.302+0000 7f3a0bfff640 -1 JournalMetadata: 0x55e666d35990 failed to locate client: INSPECT 2024-09-21T23:39:17.311 INFO:tasks.workunit.client.0.smithi165.stderr:+ queue_length=0 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ tries=4 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 4 -lt 10 ']' 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 0 -gt 0 ']' 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 0 -eq 0 ']' 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd2 24419 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd2 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=24419 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:17.312 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:17.313 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:17.313 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:17.315 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:17.315 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:17.315 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:17.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:17.385 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:17.385+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:17.418 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:17.419 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:17.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:17.433 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:17.433 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^24419\b' 2024-09-21T23:39:17.433 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^24419\b' 2024-09-21T23:39:17.480 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:17.482+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:17.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:17.490 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:17.496 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:17.497 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:17.497 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *24419\b' 2024-09-21T23:39:17.497 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *24419\b' 2024-09-21T23:39:17.505 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:17.505 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:17.507 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:17.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:17.523 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:17.523 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:17.543 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:17.543 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:17.551 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:17.551 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:17.564 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:17.564 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:17.565 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:17.576 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:17.577+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:17.583 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:17.583 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:17.677 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:17.677+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:17.713 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:39:17.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:39:17.728 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:17.728 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:17.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:17.728 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:17.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.364756857000003, 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:39:17.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.353092818, 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:39:17.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:39:17.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:39:17.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.341526588999997, 2024-09-21T23:39:17.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:700 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'30, mlcod=54'28)", 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.855589053000003, 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028768000000000001, 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872689+0000", 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.872690+0000", 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3900000000000003e-07 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873123+0000", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043362899999999999 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873140+0000", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6180000000000001e-05 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873147+0000", 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6729999999999992e-06 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873217+0000", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9647999999999995e-05 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.873249+0000", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2073999999999998e-05 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875519+0000", 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022696309999999998 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.875566+0000", 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7026e-05 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:706 2.7 e55/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 55'36, mlcod=55'34)", 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.831678936000003, 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002873282, 2024-09-21T23:39:17.746 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896599+0000", 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.896600+0000", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1880000000000001e-06 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897002+0000", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040175299999999997 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897018+0000", 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6302000000000001e-05 2024-09-21T23:39:17.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897028+0000", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4169999999999998e-06 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897099+0000", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1310999999999995e-05 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.897124+0000", 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5516000000000001e-05 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899275+0000", 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021504129999999999 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.899472+0000", 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019738199999999999 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.750 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.823187875000002, 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.754 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.804052290999998, 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:39:17.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:39:17.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:39:17.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.794647971000003, 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:39:17.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:17.763 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.149212845000001, 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:39:17.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.117558461999998, 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:39:17.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:39:17.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.113855387999998, 2024-09-21T23:39:17.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:39:17.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:39:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:39:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:39:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:39:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:53 2.7 e58/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 58'85, mlcod=58'83)", 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.558251959000003, 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028844119999999998, 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170026+0000", 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170027+0000", 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9500000000000001e-07 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170414+0000", 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038727100000000001 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170432+0000", 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7479999999999999e-05 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170445+0000", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3189e-05 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170503+0000", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7812999999999997e-05 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.170517+0000", 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3971000000000001e-05 2024-09-21T23:39:17.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172870+0000", 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023534770000000001 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.172910+0000", 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0315999999999998e-05 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.099704707999997, 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:39:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.088341450000001, 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:39:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:39:17.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:17.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.076946493000001, 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:17.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.795 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.067337754, 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:17.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:39:17.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:39:17.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:32 2.7 e75/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 75'154, mlcod=75'151)", 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.211981722000001, 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010506980000000001, 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.800 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516298+0000", 2024-09-21T23:39:17.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0040000000000002e-06 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516317+0000", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9060999999999999e-05 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516356+0000", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8831999999999997e-05 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516380+0000", 2024-09-21T23:39:17.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4139999999999999e-05 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516532+0000", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015149900000000001 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516573+0000", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1007999999999998e-05 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526749+0000", 2024-09-21T23:39:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010175797 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526803+0000", 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639000000000003e-05 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:60 2.7 e76/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 76'156, mlcod=75'152)", 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.158956319, 2024-09-21T23:39:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071394079999999999, 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999996e-07 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569332+0000", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0203e-05 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569361+0000", 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8127999999999999e-05 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569390+0000", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9263000000000001e-05 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569468+0000", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8051000000000002e-05 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569493+0000", 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5313e-05 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576410+0000", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069164969999999997 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576461+0000", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1348999999999999e-05 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:64 2.7 e76/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 76'158, mlcod=75'152)", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.151265801999999, 2024-09-21T23:39:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042077199999999999, 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577014+0000", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.612e-06 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577027+0000", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3570999999999999e-05 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577054+0000", 2024-09-21T23:39:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6713999999999999e-05 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577070+0000", 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577138+0000", 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8003999999999995e-05 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577157+0000", 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9587000000000001e-05 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581148+0000", 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039903500000000001 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581220+0000", 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2174000000000004e-05 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5254.0:37 2.7 e77/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 77'184, mlcod=77'180)", 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2519486890000007, 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029648180000000001, 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476330+0000", 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476454+0000", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012381499999999999 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476468+0000", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3737000000000001e-05 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476477+0000", 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477554+0000", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010771789999999999 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477568+0000", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577000000000001e-05 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479264+0000", 2024-09-21T23:39:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001696637 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479294+0000", 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9686999999999999e-05 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:17.817 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:17.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:17.819 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:17.817+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:17.919 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:17.917+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:17.944 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:17.944 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:17.945 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:17.945 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:17.958 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:18.006 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:18.006 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd feature disable rbd/testrbdnbd21555 journaling 2024-09-21T23:39:18.020 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.018+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.034 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:39:18.038 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:39:18.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:39:18.086 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:18.086 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:18.086 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:18.086 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:18.086 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:18.101 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:18.110 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd config image rm rbd/testrbdnbd21555 rbd_discard_granularity_bytes 2024-09-21T23:39:18.149 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.150+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.187 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:18.187 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:18.187 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:18.187 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:18.188 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:18.189 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:18.189 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:18.190 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:18.221 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.222+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.268 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:18.268 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:18.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:18.268 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.171524501999997, 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:39:18.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:39:18.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:39:18.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:39:18.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.157446954999998, 2024-09-21T23:39:18.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.148355144, 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:39:18.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.126308047999999, 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:18.285 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:18.290 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.022932716, 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:18.294 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.022803578000001, 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:39:18.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.990999160999998, 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:18.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:39:18.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:39:18.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:34 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.748210049000001, 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023817232000000001, 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.303 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 34 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518869+0000", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518884+0000", 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4742e-05 2024-09-21T23:39:18.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518940+0000", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5745999999999998e-05 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518996+0000", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5943999999999998e-05 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519161+0000", 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016499700000000001 2024-09-21T23:39:18.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519412+0000", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025155899999999999 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519930+0000", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051799400000000003 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.537789+0000", 2024-09-21T23:39:18.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017859223 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542572+0000", 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004782171 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542635+0000", 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3083999999999994e-05 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542684+0000", 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9135999999999998e-05 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:36 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.748016472, 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024223537999999999, 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:18.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519061+0000", 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.283e-06 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519069+0000", 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7190000000000001e-06 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519094+0000", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4904000000000001e-05 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519111+0000", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7251000000000001e-05 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529102+0000", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099906300000000003 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529230+0000", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012755500000000001 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529483+0000", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025336899999999998 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.539134+0000", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096515980000000008 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542876+0000", 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003741439 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543167+0000", 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029152799999999999 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543284+0000", 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011626200000000001 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:59 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.698918683, 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018089152000000001, 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 59 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568160+0000", 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9589999999999998e-06 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568169+0000", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0129999999999999e-06 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568192+0000", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-05 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568207+0000", 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5804e-05 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568324+0000", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116344 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568338+0000", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4154e-05 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571148+0000", 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028103519999999999 2024-09-21T23:39:18.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571230+0000", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-05 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571406+0000", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017574999999999999 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576139+0000", 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047326759999999999 2024-09-21T23:39:18.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586043+0000", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099038500000000005 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586151+0000", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108323 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586247+0000", 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6277e-05 2024-09-21T23:39:18.320 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:92 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2125824~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.659095334, 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015425227, 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.321 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 92 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607982+0000", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4900000000000001e-07 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608111+0000", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012953800000000001 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608118+0000", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5180000000000001e-06 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608121+0000", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4989999999999999e-06 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614540+0000", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064183850000000004 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614587+0000", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7521000000000001e-05 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614665+0000", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7497999999999996e-05 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617185+0000", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025206109999999999 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623310+0000", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006124694 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623355+0000", 2024-09-21T23:39:18.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4842000000000002e-05 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623407+0000", 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1771999999999998e-05 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:93 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2256896~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658949502, 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015447901, 2024-09-21T23:39:18.326 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 93 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.41e-07 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608256+0000", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128291 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608264+0000", 2024-09-21T23:39:18.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7740000000000004e-06 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608269+0000", 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2279999999999998e-06 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615063+0000", 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067938929999999996 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615159+0000", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6428999999999994e-05 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615288+0000", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012889800000000001 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617455+0000", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021670169999999998 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623474+0000", 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060186069999999996 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623520+0000", 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6347999999999997e-05 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623575+0000", 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5074999999999997e-05 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:94 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2387968~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658796681, 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017672687999999999, 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 94 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608281+0000", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7800000000000001e-07 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608423+0000", 2024-09-21T23:39:18.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142819 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608433+0000", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.38e-06 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608440+0000", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615922+0000", 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074819919999999998 2024-09-21T23:39:18.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615968+0000", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6622000000000002e-05 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616051+0000", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2434000000000004e-05 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617461+0000", 2024-09-21T23:39:18.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014100060000000001 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625814+0000", 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083532390000000002 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625896+0000", 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1805999999999996e-05 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625953+0000", 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6932000000000003e-05 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:95 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2519040~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658625761, 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017624719, 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:18.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608452+0000", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608524+0000", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2565999999999996e-05 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608532+0000", 2024-09-21T23:39:18.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2879999999999993e-06 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608539+0000", 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616439+0000", 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007900041 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616540+0000", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010035999999999999 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616676+0000", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0001362 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619529+0000", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028531250000000002 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625999+0000", 2024-09-21T23:39:18.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064702960000000004 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626029+0000", 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9702000000000001e-05 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626076+0000", 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6656000000000002e-05 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:96 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2650112~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658523772000001, 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017659136999999998, 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:18.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608620+0000", 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6256999999999999e-05 2024-09-21T23:39:18.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608634+0000", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608640+0000", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1659999999999999e-06 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617495+0000", 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088556810000000007 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617544+0000", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8162999999999999e-05 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617618+0000", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4759999999999996e-05 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619535+0000", 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001916374 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626121+0000", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065866299999999996 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626166+0000", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4202999999999997e-05 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626212+0000", 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6434000000000003e-05 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:97 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2781184~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658426577, 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017675217999999999, 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 97 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608651+0000", 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9299999999999998e-07 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608730+0000", 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9598000000000006e-05 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608739+0000", 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0119999999999994e-06 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608746+0000", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2349999999999996e-06 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618001+0000", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092555099999999998 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618092+0000", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0552999999999999e-05 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618235+0000", 2024-09-21T23:39:18.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014355499999999999 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620378+0000", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021429750000000001 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626257+0000", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0058784980000000002 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626283+0000", 2024-09-21T23:39:18.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.601e-05 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626325+0000", 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2679000000000003e-05 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:98 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2912256~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658321021999999, 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017700770000000001, 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.353 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 98 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608873+0000", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117304 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608887+0000", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3386e-05 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608894+0000", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618777+0000", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098831249999999995 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618825+0000", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8563000000000002e-05 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618902+0000", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6130000000000005e-05 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621131+0000", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022290230000000001 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626371+0000", 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052399329999999996 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626412+0000", 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1844999999999997e-05 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626456+0000", 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4013e-05 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:99 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3043328~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658171359000001, 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017665733999999999, 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 99 2024-09-21T23:39:18.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608906+0000", 2024-09-21T23:39:18.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608993+0000", 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6874999999999997e-05 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609004+0000", 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0849999999999999e-05 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609010+0000", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619684+0000", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010673867 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619760+0000", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6249999999999997e-05 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619880+0000", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011974200000000001 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622034+0000", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021542639999999999 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626501+0000", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044669829999999999 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626528+0000", 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626571+0000", 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3123e-05 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:100 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3174400~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:18.364 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.658023056999999, 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01765369, 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:18.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609143+0000", 2024-09-21T23:39:18.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9022000000000002e-05 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609153+0000", 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8160000000000005e-06 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609161+0000", 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3420000000000008e-06 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620487+0000", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011325052 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620536+0000", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654999999999998e-05 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620615+0000", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8403000000000001e-05 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622168+0000", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015539270000000001 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626619+0000", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044508730000000002 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626662+0000", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2175000000000002e-05 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626707+0000", 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5741999999999997e-05 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:101 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3305472~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.657902356999999, 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020329684000000001, 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:18.370 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:18.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609175+0000", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2200000000000004e-07 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609264+0000", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8688000000000002e-05 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609273+0000", 2024-09-21T23:39:18.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0380000000000005e-06 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609277+0000", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.036e-06 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621290+0000", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012013429000000001 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621385+0000", 2024-09-21T23:39:18.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5260000000000006e-05 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621524+0000", 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013853100000000001 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622385+0000", 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00086160000000000002 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:18.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629426+0000", 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007040129 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629455+0000", 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9274000000000001e-05 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629504+0000", 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9076999999999998e-05 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.375 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.376 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.376 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:18.376 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:18.376 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:18.377 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd2 2024-09-21T23:39:18.377 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:18.378 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:18.378 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:18.378 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:18.378 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd2'\'']/id' 2024-09-21T23:39:18.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:18.379 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:18.377+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:18.430 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24855 2024-09-21T23:39:18.430 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24855 2024-09-21T23:39:18.430 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24855 -C rbd-nbd 2024-09-21T23:39:18.435 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:18.435 INFO:tasks.workunit.client.0.smithi165.stdout: 24855 ? 00:00:00 rbd-nbd 2024-09-21T23:39:18.436 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd feature enable rbd/testrbdnbd21555 journaling 2024-09-21T23:39:18.478 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.478+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.499 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:18.500 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:18.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:18.536 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-21T23:39:18.536 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:18.537 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:18.538 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:18.538 INFO:tasks.workunit.client.0.smithi165.stderr:++ which blkdiscard 2024-09-21T23:39:18.540 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/sbin/blkdiscard 2024-09-21T23:39:18.541 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:18.541 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/sbin/blkdiscard --offset 0 --length 4096 /dev/nbd2 2024-09-21T23:39:18.579 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.578+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.613 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd feature disable rbd/testrbdnbd21555 journaling 2024-09-21T23:39:18.629 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:18.629 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:18.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:18.680 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.678+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:18.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd2 24855 2024-09-21T23:39:18.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd2 2024-09-21T23:39:18.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=24855 2024-09-21T23:39:18.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:18.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:18.712 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:18.713 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:18.714 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:18.715 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:18.715 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:18.715 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.14162512599999, 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.13757304400001, 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:138 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'87)", 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.136759384, 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046525840000000004, 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633128+0000", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633129+0000", 2024-09-21T23:39:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633143+0000", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4336e-05 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633165+0000", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1844e-05 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633180+0000", 2024-09-21T23:39:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983e-05 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634380+0000", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012004369999999999 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.634422+0000", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1829999999999998e-05 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637743+0000", 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033204670000000001 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637781+0000", 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7759999999999998e-05 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.092342727999998, 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:39:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.084962601000001, 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:39:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:39:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:676 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'36, mlcod=54'34)", 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.037238578999997, 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046501720000000002, 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732649+0000", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.732651+0000", 2024-09-21T23:39:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733544+0000", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089316999999999999 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733573+0000", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8863000000000001e-05 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733586+0000", 2024-09-21T23:39:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.344e-05 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733759+0000", 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017232599999999999 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.733795+0000", 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6569000000000002e-05 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737191+0000", 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033952380000000001 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.737299+0000", 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010836300000000001 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:678 2.5 e54/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 54'38, mlcod=54'36)", 2024-09-21T23:39:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025244035999997, 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004657942, 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744643+0000", 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.744646+0000", 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2299999999999998e-06 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745565+0000", 2024-09-21T23:39:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000919647 2024-09-21T23:39:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745594+0000", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8762999999999999e-05 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745607+0000", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3563e-05 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745788+0000", 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018041800000000001 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.745828+0000", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9777000000000002e-05 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749223+0000", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033955750000000001 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.749301+0000", 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7968999999999998e-05 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:110 2.5 e58/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 58'97, mlcod=58'95)", 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.341553916999999, 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047648980000000001, 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428333+0000", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.428335+0000", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.19e-06 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429762+0000", 2024-09-21T23:39:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014277230000000001 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429786+0000", 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3427000000000001e-05 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429803+0000", 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7093000000000001e-05 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429914+0000", 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111606 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.429953+0000", 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8816000000000002e-05 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433012+0000", 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030585500000000002 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.433098+0000", 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6493e-05 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.293880434999998, 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.270876231000003, 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:39:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:39:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.257038149000003, 2024-09-21T23:39:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:39:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:39:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:25 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'135, mlcod=75'133)", 2024-09-21T23:39:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.262935933, 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005751763, 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506953+0000", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506965+0000", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-05 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506993+0000", 2024-09-21T23:39:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8028999999999999e-05 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507006+0000", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3417e-05 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507142+0000", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013520200000000001 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507195+0000", 2024-09-21T23:39:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3117e-05 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512636+0000", 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054415619999999996 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512703+0000", 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6863e-05 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:26 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'136, mlcod=75'133)", 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.262246319000001, 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051681380000000001, 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507643+0000", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.773e-06 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507658+0000", 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5136999999999999e-05 2024-09-21T23:39:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507687+0000", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9306999999999999e-05 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507701+0000", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3376e-05 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508223+0000", 2024-09-21T23:39:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052228599999999995 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508266+0000", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3037000000000001e-05 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512707+0000", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044412159999999996 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512809+0000", 2024-09-21T23:39:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102006 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:28 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'138, mlcod=75'133)", 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.261000748000001, 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068616110000000001, 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508888+0000", 2024-09-21T23:39:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.753e-06 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508902+0000", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3543999999999999e-05 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508928+0000", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5809e-05 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508944+0000", 2024-09-21T23:39:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6390000000000001e-05 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510106+0000", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001162185 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510155+0000", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8912000000000002e-05 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515650+0000", 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054947529999999998 2024-09-21T23:39:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515748+0000", 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8264999999999995e-05 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:29 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'139, mlcod=75'133)", 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.260378490000001, 2024-09-21T23:39:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063237359999999999, 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509511+0000", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.747e-06 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509528+0000", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7047000000000001e-05 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509559+0000", 2024-09-21T23:39:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1463e-05 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509571+0000", 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2047e-05 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510923+0000", 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001351882 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510973+0000", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9767e-05 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515755+0000", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00478269 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515833+0000", 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7093000000000002e-05 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:30 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'140, mlcod=75'133)", 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.259705281, 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057255370000000002, 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510184+0000", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9360000000000002e-06 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510199+0000", 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4564e-05 2024-09-21T23:39:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510225+0000", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510250+0000", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4323000000000001e-05 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511937+0000", 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016875010000000001 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511974+0000", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7425999999999997e-05 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515836+0000", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038614370000000001 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515908+0000", 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1717000000000005e-05 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:47 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'149, mlcod=75'144)", 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.224616041000001, 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067526640000000002, 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545272+0000", 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6099999999999999e-07 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545279+0000", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4899999999999997e-06 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545293+0000", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3889e-05 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545301+0000", 2024-09-21T23:39:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1980000000000001e-06 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545364+0000", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2890000000000003e-05 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545383+0000", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9074999999999999e-05 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.551967+0000", 2024-09-21T23:39:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065840680000000002 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552024+0000", 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7192999999999999e-05 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:48 2.5 e76/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 76'150, mlcod=75'147)", 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.222843048, 2024-09-21T23:39:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050129090000000003, 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547046+0000", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2470000000000001e-06 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547053+0000", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6029999999999999e-06 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547069+0000", 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5863e-05 2024-09-21T23:39:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547077+0000", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4239999999999993e-06 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547146+0000", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8106000000000001e-05 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547170+0000", 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4871999999999998e-05 2024-09-21T23:39:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552029+0000", 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048586669999999997 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552057+0000", 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8127e-05 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:55 2.5 e76/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 76'159, mlcod=76'153)", 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.212111306000001, 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065210290000000002, 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557781+0000", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.096e-06 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557790+0000", 2024-09-21T23:39:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1499999999999999e-06 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557794+0000", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3749999999999996e-06 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557832+0000", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8475e-05 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557845+0000", 2024-09-21T23:39:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2607e-05 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564252+0000", 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064069690000000002 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564297+0000", 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5015000000000002e-05 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5868.0:7 2.4 2:2ff85688:::journal_data.2.122ca41dcc05.0:head [read 0~78 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.6155091029999999, 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55823445699999996, 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5868", 2024-09-21T23:39:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/1315715008", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154380+0000", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154386+0000", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5470000000000004e-06 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154397+0000", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1581e-05 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154409+0000", 2024-09-21T23:39:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.132e-05 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154486+0000", 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7667e-05 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154497+0000", 2024-09-21T23:39:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1073000000000001e-05 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.710968+0000", 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55647108099999998 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.711233+0000", 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026443799999999998 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.712613+0000", 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00137992 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:18.856 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:18.858 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:18.858 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^24855\b' 2024-09-21T23:39:18.858 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^24855\b' 2024-09-21T23:39:18.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:18.859 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:18.857+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:18.903 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:18.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:18.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *24855\b' 2024-09-21T23:39:18.904 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *24855\b' 2024-09-21T23:39:18.911 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:18.911 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:18.958 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:18.958+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:19.032 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:19.032 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:19.032 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:19.032 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:19.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:19.051 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:39:19.058 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:19.057+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:19.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:39:19.160 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:19.161+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:19.166 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:19.166 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:19.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:19.260 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:19.262+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.940454889999998, 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:39:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.929195548000003, 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:39:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:39:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.907586414000001, 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:39:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:39:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.861490926999998, 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:39:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.84946884, 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:39:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:39:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:39:19.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:19.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.828171900000001, 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:39:19.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:39:19.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.817353437999998, 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:39:19.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:39:19.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.100242876999999, 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:39:19.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:39:19.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.100150048000003, 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:19.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:39:19.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:39:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:39:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:39:19.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.763325899000002, 2024-09-21T23:39:19.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:39:19.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:39:19.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.742173526000002, 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:39:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:39:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:39:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.155942695, 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:39:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:39:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.020613167, 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:39:19.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.795564204000001, 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:39:19.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:39:19.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:33 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.827416818, 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019445529, 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:19.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515329+0000", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515340+0000", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0482999999999999e-05 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515363+0000", 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3193999999999998e-05 2024-09-21T23:39:19.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515379+0000", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5996000000000001e-05 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515512+0000", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132943 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515745+0000", 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023335399999999999 2024-09-21T23:39:19.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516012+0000", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026738199999999998 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534312+0000", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018299214000000001 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534516+0000", 2024-09-21T23:39:19.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020432600000000001 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534653+0000", 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013709200000000001 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534772+0000", 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118942 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:35 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.826045595, 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018667975, 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516708+0000", 2024-09-21T23:39:19.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0077e-05 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516719+0000", 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1683e-05 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516750+0000", 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0309000000000001e-05 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516773+0000", 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2762999999999999e-05 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523637+0000", 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068647480000000004 2024-09-21T23:39:19.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523818+0000", 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077600000000001 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.524105+0000", 2024-09-21T23:39:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028720500000000001 2024-09-21T23:39:19.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534471+0000", 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010365740999999999 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535149+0000", 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067791800000000001 2024-09-21T23:39:19.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535270+0000", 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120932 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535366+0000", 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5822999999999994e-05 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:34 2.2 e75/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 75'43, mlcod=58'40)", 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.824205879000001, 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022102816000000001, 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518540+0000", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518554+0000", 2024-09-21T23:39:19.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4326e-05 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518580+0000", 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5403000000000001e-05 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518596+0000", 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518725+0000", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012880300000000001 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518759+0000", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4335999999999998e-05 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540576+0000", 2024-09-21T23:39:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021817020999999999 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540640+0000", 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3917000000000005e-05 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.5215.0:33 2.6 e75/17 ondisk, result = 0)", 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.821089276, 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012683450000000001, 2024-09-21T23:39:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521656+0000", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521668+0000", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2092999999999999e-05 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521695+0000", 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6503999999999999e-05 2024-09-21T23:39:19.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521714+0000", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9150000000000001e-05 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534274+0000", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012560135 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534295+0000", 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1118999999999999e-05 2024-09-21T23:39:19.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534337+0000", 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2466000000000003e-05 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:36 2.2 e75/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 75'45, mlcod=58'40)", 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.813823411, 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0117643, 2024-09-21T23:39:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528922+0000", 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6129999999999998e-06 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528940+0000", 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7283e-05 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529003+0000", 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3319999999999997e-05 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529024+0000", 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.092e-05 2024-09-21T23:39:19.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529224+0000", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019950499999999999 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529338+0000", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114429 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540655+0000", 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011317117 2024-09-21T23:39:19.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540684+0000", 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9113000000000001e-05 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:57 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.784434148999999, 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013068644000000001, 2024-09-21T23:39:19.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 57 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558312+0000", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558326+0000", 2024-09-21T23:39:19.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3668e-05 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558359+0000", 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3534000000000001e-05 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558376+0000", 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6137000000000002e-05 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558480+0000", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104811 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558622+0000", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014147699999999999 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558881+0000", 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025933899999999998 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564172+0000", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052903009999999999 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571263+0000", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070919110000000002 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571332+0000", 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8733999999999996e-05 2024-09-21T23:39:19.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571378+0000", 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5629e-05 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:19.471 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:19.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:++ ceph-conf --show-config-value librados_thread_count 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:+ EXPECTED=2 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:19.474 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:19.475 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:19.475 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd map rbd/testrbdnbd21555 2024-09-21T23:39:19.475 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:19.474+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:19.573 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:19.573+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:19.593 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd2 2024-09-21T23:39:19.593 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:19.593 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:19.593 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:19.593 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:19.594 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd2'\'']/id' 2024-09-21T23:39:19.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:19.603 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:19.603 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:19.603 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:19.603 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:19.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:19.691 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=24966 2024-09-21T23:39:19.692 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 24966 2024-09-21T23:39:19.692 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 24966 -C rbd-nbd 2024-09-21T23:39:19.699 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:19.699 INFO:tasks.workunit.client.0.smithi165.stdout: 24966 ? 00:00:00 rbd-nbd 2024-09-21T23:39:19.700 INFO:tasks.workunit.client.0.smithi165.stderr:++ ps -p 24966 -T 2024-09-21T23:39:19.700 INFO:tasks.workunit.client.0.smithi165.stderr:++ grep -c io_context_pool 2024-09-21T23:39:19.708 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:19.709+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:19.709 INFO:tasks.workunit.client.0.smithi165.stderr:+ ACTUAL=2 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 2 -eq 2 ']' 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd2 24966 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd2 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=24966 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:19.710 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:19.711 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:19.711 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:19.712 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:19.712 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:19.712 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:19.754 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:19.770 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:19.770 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:19.774 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:19.773+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:19.782 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:19.782 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:19.788 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:19.788 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:19.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:19.809 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:19.809 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:19.820 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:19.820 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^24966\b' 2024-09-21T23:39:19.821 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^24966\b' 2024-09-21T23:39:19.830 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:19.831 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:19.851 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:19.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:19.880 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:19.881 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:19.881 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *24966\b' 2024-09-21T23:39:19.881 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *24966\b' 2024-09-21T23:39:19.888 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:19.888 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:19.908 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:19.909+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:19.923 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:19.923 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:19.936 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:19.976 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:19.977+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:20.055 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:39:20.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:39:20.069 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.069 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:39:20.077 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:20.078+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:20.084 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:20.177 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:20.178+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:20.249 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.249 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.264 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:20.278 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:20.277+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:20.379 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:20.378+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:20.389 INFO:tasks.workunit.client.0.smithi165.stderr:+ EXPECTED=7 2024-09-21T23:39:20.390 INFO:tasks.workunit.client.0.smithi165.stderr:++ _sudo rbd device --device-type nbd --rbd-op-threads 7 map rbd/testrbdnbd21555 2024-09-21T23:39:20.390 INFO:tasks.workunit.client.0.smithi165.stderr:++ local cmd 2024-09-21T23:39:20.390 INFO:tasks.workunit.client.0.smithi165.stderr:+++ id -u 2024-09-21T23:39:20.391 INFO:tasks.workunit.client.0.smithi165.stderr:++ '[' 1000 -eq 0 ']' 2024-09-21T23:39:20.392 INFO:tasks.workunit.client.0.smithi165.stderr:+++ which rbd 2024-09-21T23:39:20.394 INFO:tasks.workunit.client.0.smithi165.stderr:++ cmd=/usr/bin/rbd 2024-09-21T23:39:20.394 INFO:tasks.workunit.client.0.smithi165.stderr:++ shift 2024-09-21T23:39:20.394 INFO:tasks.workunit.client.0.smithi165.stderr:++ sudo -nE /usr/bin/rbd device --device-type nbd --rbd-op-threads 7 map rbd/testrbdnbd21555 2024-09-21T23:39:20.433 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.433 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.447 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:20.489 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:20.490+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:20.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV=/dev/nbd2 2024-09-21T23:39:20.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ get_pid rbd 2024-09-21T23:39:20.549 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pool=rbd 2024-09-21T23:39:20.550 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns= 2024-09-21T23:39:20.550 INFO:tasks.workunit.client.0.smithi165.stderr:++ rbd device --device-type nbd --format xml list 2024-09-21T23:39:20.550 INFO:tasks.workunit.client.0.smithi165.stderr:++ xmlstarlet sel -t -v '//devices/device[pool='\''rbd'\''][namespace='\'''\''][image='\''testrbdnbd21555'\''][device='\''/dev/nbd2'\'']/id' 2024-09-21T23:39:20.581 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:20.581+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:20.608 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.608 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.625 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:20.648 INFO:tasks.workunit.client.0.smithi165.stderr:+ PID=25031 2024-09-21T23:39:20.648 INFO:tasks.workunit.client.0.smithi165.stderr:+ test -n 25031 2024-09-21T23:39:20.648 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -p 25031 -C rbd-nbd 2024-09-21T23:39:20.656 INFO:tasks.workunit.client.0.smithi165.stdout: PID TTY TIME CMD 2024-09-21T23:39:20.656 INFO:tasks.workunit.client.0.smithi165.stdout: 25031 ? 00:00:00 rbd-nbd 2024-09-21T23:39:20.657 INFO:tasks.workunit.client.0.smithi165.stderr:++ ps -p 25031 -T 2024-09-21T23:39:20.657 INFO:tasks.workunit.client.0.smithi165.stderr:++ grep -c io_context_pool 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ ACTUAL=7 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 7 -eq 7 ']' 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ unmap_device /dev/nbd2 25031 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ local args=/dev/nbd2 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ local pid=25031 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ _sudo rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:20.665 INFO:tasks.workunit.client.0.smithi165.stderr:+ local cmd 2024-09-21T23:39:20.666 INFO:tasks.workunit.client.0.smithi165.stderr:++ id -u 2024-09-21T23:39:20.667 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' 1000 -eq 0 ']' 2024-09-21T23:39:20.667 INFO:tasks.workunit.client.0.smithi165.stderr:++ which rbd 2024-09-21T23:39:20.668 INFO:tasks.workunit.client.0.smithi165.stderr:+ cmd=/usr/bin/rbd 2024-09-21T23:39:20.668 INFO:tasks.workunit.client.0.smithi165.stderr:+ shift 2024-09-21T23:39:20.669 INFO:tasks.workunit.client.0.smithi165.stderr:+ sudo -nE /usr/bin/rbd device --device-type nbd unmap /dev/nbd2 2024-09-21T23:39:20.682 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:20.681+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:20.747 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.747 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.762 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:20.783 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd device --device-type nbd list 2024-09-21T23:39:20.784 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^25031\b' 2024-09-21T23:39:20.784 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^25031\b' 2024-09-21T23:39:20.809 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:20.810+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:20.843 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:20.845 INFO:tasks.workunit.client.0.smithi165.stderr:+ ps -C rbd-nbd 2024-09-21T23:39:20.845 INFO:tasks.workunit.client.0.smithi165.stderr:+ expect_false grep '^ *25031\b' 2024-09-21T23:39:20.846 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep '^ *25031\b' 2024-09-21T23:39:20.849 INFO:tasks.workunit.client.0.smithi165.stderr:+ return 0 2024-09-21T23:39:20.850 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:20.883 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:20.882+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:20.886 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:20.886 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:20.900 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:20.993 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:20.994+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:21.064 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:39:21.065 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.065 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.069 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:39:21.081 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:21.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:39:21.129 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:21.130+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:21.192 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:21.193+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:21.251 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.251 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.265 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:21.309 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:21.310+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:21.351 INFO:tasks.workunit.client.0.smithi165.stderr:+ DEV= 2024-09-21T23:39:21.351 INFO:tasks.workunit.client.0.smithi165.stdout:OK 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ echo OK 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ cleanup 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ local ns s 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ set +e 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ mount 2024-09-21T23:39:21.352 INFO:tasks.workunit.client.0.smithi165.stderr:+ fgrep /tmp/tmp.h0CUjCtN4b/mnt 2024-09-21T23:39:21.361 INFO:tasks.workunit.client.0.smithi165.stderr:+ rm -Rf /tmp/tmp.h0CUjCtN4b 2024-09-21T23:39:21.374 INFO:tasks.workunit.client.0.smithi165.stderr:+ '[' -n '' ']' 2024-09-21T23:39:21.375 INFO:tasks.workunit.client.0.smithi165.stderr:+ for ns in '' ${NS} 2024-09-21T23:39:21.375 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace '' status testrbdnbd21555 2024-09-21T23:39:21.384 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.385 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.387 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:21.385+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:21.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:21.431 INFO:tasks.workunit.client.0.smithi165.stdout:Watchers: none 2024-09-21T23:39:21.435 INFO:tasks.workunit.client.0.smithi165.stderr:+ for s in 0.5 1 2 4 8 16 32 2024-09-21T23:39:21.435 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:21.487 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:21.485+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:21.517 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.518 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:21.616 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:21.617+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:21.690 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:21.690+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:21.696 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.696 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:21.791 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:21.790+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:21.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:21.875 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:21.875 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:21.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:21.937 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:21.937+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:21.937 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace '' status testrbdnbd21555 2024-09-21T23:39:21.937 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep 'Watchers: none' 2024-09-21T23:39:21.992 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:21.990+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:22.001 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:22.002 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:22.002 INFO:tasks.workunit.client.0.smithi165.stdout:Watchers: none 2024-09-21T23:39:22.003 INFO:tasks.workunit.client.0.smithi165.stderr:+ break 2024-09-21T23:39:22.003 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace '' snap purge testrbdnbd21555 2024-09-21T23:39:22.018 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:22.018 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:22.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:22.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:22.054 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:22.054 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:22.064 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:22.064 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:22.070 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:39:22.079 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:22.079 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:22.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:22.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:39:22.093 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:22.094+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:22.101 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:22.194 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:22.194+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:22.197 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:22.198 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:22.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:22.295 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:22.293+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:620 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.920181491999998, 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070304369999999996, 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 620 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402288+0000", 2024-09-21T23:39:22.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.402290+0000", 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404178+0000", 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018876489999999999 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404209+0000", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1393000000000003e-05 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404224+0000", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.451e-05 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404416+0000", 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019232900000000001 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404563+0000", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014637499999999999 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.404777+0000", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021487200000000001 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.408462+0000", 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036845950000000001 2024-09-21T23:39:22.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409150+0000", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068795499999999997 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409253+0000", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102756 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.409319+0000", 2024-09-21T23:39:22.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6050000000000006e-05 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:622 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.908922150000002, 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062824359999999997, 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 622 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413548+0000", 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.413550+0000", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414590+0000", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010397620000000001 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414623+0000", 2024-09-21T23:39:22.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3553000000000002e-05 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414640+0000", 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6192999999999999e-05 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414826+0000", 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186076 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.414964+0000", 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013796899999999999 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.415188+0000", 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022447799999999999 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.418529+0000", 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033409149999999999 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419602+0000", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001072845 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419758+0000", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156062 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.419830+0000", 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2253000000000001e-05 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:626 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.887313016, 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067057230000000002, 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 626 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435157+0000", 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.435159+0000", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436206+0000", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010468020000000001 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436239+0000", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3173000000000002e-05 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436255+0000", 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5546999999999999e-05 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436402+0000", 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014753499999999999 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436539+0000", 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000136923 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.436743+0000", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000203585 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.440750+0000", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040071799999999999 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441672+0000", 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092247300000000004 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441757+0000", 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4228999999999996e-05 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.441863+0000", 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106016 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:634 2.2 e54/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 54'15, mlcod=54'13)", 2024-09-21T23:39:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.841217528999998, 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063581760000000001, 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481252+0000", 2024-09-21T23:39:22.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.481254+0000", 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3570000000000001e-06 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482173+0000", 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091883299999999998 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482197+0000", 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4737e-05 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482210+0000", 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2846e-05 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482320+0000", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110189 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.482358+0000", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7846999999999999e-05 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487522+0000", 2024-09-21T23:39:22.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051634539999999996 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.487611+0000", 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8912999999999994e-05 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:636 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 131072~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.829195442, 2024-09-21T23:39:22.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065947970000000003, 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 636 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:22.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493274+0000", 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.493284+0000", 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2329999999999995e-06 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494701+0000", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001417303 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494734+0000", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2790000000000003e-05 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494749+0000", 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5036e-05 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.494895+0000", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146129 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495058+0000", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016278199999999999 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.495264+0000", 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020663499999999999 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.498949+0000", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003684976 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499712+0000", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076290400000000001 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499801+0000", 2024-09-21T23:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8676000000000003e-05 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.499869+0000", 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8332999999999998e-05 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:640 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.807898502, 2024-09-21T23:39:22.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058609559999999996, 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 640 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514571+0000", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.514574+0000", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.193e-06 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515548+0000", 2024-09-21T23:39:22.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097418600000000004 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515581+0000", 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2750000000000003e-05 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515595+0000", 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4823e-05 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515737+0000", 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014212900000000001 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.515873+0000", 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135716 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.516072+0000", 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000198748 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.519769+0000", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003697489 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520275+0000", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050569299999999998 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520366+0000", 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1249999999999995e-05 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.520432+0000", 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5979e-05 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4754.0:642 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [write 3276800~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.797080039999997, 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058494059999999997, 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4754", 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3802530801", 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 642 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525390+0000", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.525407+0000", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7509000000000001e-05 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526145+0000", 2024-09-21T23:39:22.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073783900000000001 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526178+0000", 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3278999999999998e-05 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526198+0000", 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9242999999999999e-05 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526393+0000", 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019514200000000001 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526532+0000", 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000139543 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.526739+0000", 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206801 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.530179+0000", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034400120000000001 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531080+0000", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090058599999999999 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531173+0000", 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3053000000000005e-05 2024-09-21T23:39:22.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.531239+0000", 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6398999999999999e-05 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:15 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.079969478999999, 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062441490000000001, 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242500+0000", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242503+0000", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242513+0000", 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0027000000000001e-05 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242536+0000", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2580000000000001e-05 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242555+0000", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.963e-05 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242692+0000", 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013660199999999999 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242850+0000", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015816800000000001 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248745+0000", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058943290000000002 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4654.0:16 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:22.367 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.079876650000003, 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012011651, 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4654", 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242593+0000", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242594+0000", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9299999999999996e-07 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242599+0000", 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4429999999999996e-06 2024-09-21T23:39:22.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242613+0000", 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4517e-05 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.242621+0000", 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3569999999999998e-06 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.248853+0000", 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062316660000000003 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.249109+0000", 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025583799999999999 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.254605+0000", 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054959370000000002 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:91 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.743052501000001, 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057015479999999999, 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 91 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579417+0000", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579419+0000", 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7269999999999999e-06 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579424+0000", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3129999999999998e-06 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579436+0000", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1846e-05 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579445+0000", 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3850000000000002e-06 2024-09-21T23:39:22.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579503+0000", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579578+0000", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4963999999999996e-05 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.579667+0000", 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8400999999999996e-05 2024-09-21T23:39:22.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.581797+0000", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021307650000000002 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585049+0000", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032511409999999999 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585087+0000", 2024-09-21T23:39:22.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8427000000000003e-05 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.585119+0000", 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1921999999999997e-05 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4856.0:107 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.721900128000001, 2024-09-21T23:39:22.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059707650000000003, 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4856", 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600570+0000", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600572+0000", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600579+0000", 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600599+0000", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9978000000000001e-05 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600608+0000", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600671+0000", 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3571000000000005e-05 2024-09-21T23:39:22.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600755+0000", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3727999999999995e-05 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.600854+0000", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8916e-05 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:22.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001902487 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606395+0000", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036387310000000001 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606476+0000", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0624000000000006e-05 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:32.606541+0000", 2024-09-21T23:39:22.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4695000000000003e-05 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:58 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [write 1179648~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.135669297, 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058638630000000004, 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:22.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 58 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186801+0000", 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.186802+0000", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188338+0000", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015359149999999999 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188360+0000", 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1931999999999999e-05 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188370+0000", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0118999999999999e-05 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188421+0000", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0416e-05 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188480+0000", 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9530000000000001e-05 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.188570+0000", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9543999999999995e-05 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.190509+0000", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001939082 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192508+0000", 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019992090000000001 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192595+0000", 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7409000000000003e-05 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.192664+0000", 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9049999999999998e-05 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:89 2.2 e58/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 58'41, mlcod=58'39)", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.000339769, 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067745269999999998, 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322130+0000", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.322132+0000", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.767e-06 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323184+0000", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001051752 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323218+0000", 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4560000000000001e-05 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323236+0000", 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8286000000000001e-05 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323356+0000", 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011990299999999999 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.323399+0000", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2713000000000003e-05 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328815+0000", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005415853 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.328905+0000", 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9692999999999997e-05 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4816.0:132 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [write 2228224~917504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.775290806000001, 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006967694, 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4816", 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/2619788023", 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 132 2024-09-21T23:39:22.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547179+0000", 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.547181+0000", 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548190+0000", 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010083620000000001 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548217+0000", 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7243e-05 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548230+0000", 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.294e-05 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548330+0000", 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9809000000000006e-05 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548465+0000", 2024-09-21T23:39:22.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013503899999999999 2024-09-21T23:39:22.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.548665+0000", 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000200562 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.551912+0000", 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032470089999999999 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.553982+0000", 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020698169999999998 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554078+0000", 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5637000000000006e-05 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.554147+0000", 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9152000000000005e-05 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:33 2.6 2:7e0a33fa:::rbd_data.122ca41dcc05.0000000000000002:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:22.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.807143419999999, 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019445529, 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515326+0000", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515329+0000", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515340+0000", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0482999999999999e-05 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515363+0000", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3193999999999998e-05 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515379+0000", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5996000000000001e-05 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515512+0000", 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132943 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515745+0000", 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023335399999999999 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516012+0000", 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026738199999999998 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534312+0000", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018299214000000001 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534516+0000", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020432600000000001 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534653+0000", 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013709200000000001 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534772+0000", 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118942 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:35 2.6 2:6acdbbc4:::rbd_data.122ca41dcc05.0000000000000004:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.805772197, 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018667975, 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516698+0000", 2024-09-21T23:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516708+0000", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0077e-05 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516719+0000", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1683e-05 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516750+0000", 2024-09-21T23:39:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0309000000000001e-05 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.516773+0000", 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2762999999999999e-05 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523637+0000", 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068647480000000004 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.523818+0000", 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077600000000001 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.524105+0000", 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028720500000000001 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534471+0000", 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010365740999999999 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535149+0000", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067791800000000001 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535270+0000", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120932 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535366+0000", 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5822999999999994e-05 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:34 2.2 e75/17 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head v 75'43, mlcod=58'40)", 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.803932481, 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022102816000000001, 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:22.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518537+0000", 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518540+0000", 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5770000000000001e-06 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518554+0000", 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4326e-05 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518580+0000", 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5403000000000001e-05 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518596+0000", 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518725+0000", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012880300000000001 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.518759+0000", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4335999999999998e-05 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540576+0000", 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021817020999999999 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540640+0000", 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3917000000000005e-05 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.5215.0:33 2.6 e75/17 ondisk, result = 0)", 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.800815878000002, 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012683450000000001, 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:22.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521654+0000", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521656+0000", 2024-09-21T23:39:22.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521668+0000", 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2092999999999999e-05 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521695+0000", 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6503999999999999e-05 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521714+0000", 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9150000000000001e-05 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534274+0000", 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012560135 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534295+0000", 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1118999999999999e-05 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.534337+0000", 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2466000000000003e-05 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:36 2.2 e75/17 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head v 75'45, mlcod=58'40)", 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.793550013000001, 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0117643, 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528920+0000", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528922+0000", 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6129999999999998e-06 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.528940+0000", 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7283e-05 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529003+0000", 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3319999999999997e-05 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529024+0000", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.092e-05 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529224+0000", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019950499999999999 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.529338+0000", 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114429 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540655+0000", 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011317117 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.540684+0000", 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9113000000000001e-05 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5215.0:57 2.6 2:644e6e2b:::rbd_data.122ca41dcc05.000000000000000a:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:22.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.764160750999999, 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013068644000000001, 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5215", 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 57 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558309+0000", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558312+0000", 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558326+0000", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3668e-05 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558359+0000", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3534000000000001e-05 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558376+0000", 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6137000000000002e-05 2024-09-21T23:39:22.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558480+0000", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104811 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558622+0000", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014147699999999999 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.558881+0000", 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025933899999999998 2024-09-21T23:39:22.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564172+0000", 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052903009999999999 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571263+0000", 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070919110000000002 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571332+0000", 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8733999999999996e-05 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.571378+0000", 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5629e-05 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.431 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:22.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:22.433 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:22.433+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:22.533 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:22.533+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:22.596 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.597 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:22.597 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:22.597 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:22.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:22.652 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:22.653+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:22.735 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:22.734+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:22.770 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:22.770 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:22.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:22.835 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:22.833+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:22.906 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:591 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'7, mlcod=48'4)", 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.604592365999999, 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077171449999999999, 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301129+0000", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301131+0000", 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301332+0000", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020117099999999999 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301359+0000", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6944000000000002e-05 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301374+0000", 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.505e-05 2024-09-21T23:39:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301786+0000", 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041193700000000002 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301829+0000", 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2616e-05 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308728+0000", 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068989180000000004 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308846+0000", 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118826 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:592 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'8, mlcod=48'4)", 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.603728793000002, 2024-09-21T23:39:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010844599, 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301993+0000", 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.301994+0000", 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302209+0000", 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021485800000000001 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302226+0000", 2024-09-21T23:39:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7183000000000001e-05 2024-09-21T23:39:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302237+0000", 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303447+0000", 2024-09-21T23:39:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001210334 2024-09-21T23:39:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303495+0000", 2024-09-21T23:39:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7477e-05 2024-09-21T23:39:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312732+0000", 2024-09-21T23:39:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092368439999999993 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312837+0000", 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105856 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:593 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'9, mlcod=48'4)", 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.603066282, 2024-09-21T23:39:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01028929, 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302655+0000", 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302657+0000", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-06 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302881+0000", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022411100000000001 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302901+0000", 2024-09-21T23:39:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0577e-05 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.302911+0000", 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304563+0000", 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001652508 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304602+0000", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999998e-05 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312851+0000", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082491479999999996 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312945+0000", 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3819000000000004e-05 2024-09-21T23:39:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:594 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'10, mlcod=48'4)", 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.602392213000002, 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097129739999999992, 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303329+0000", 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303331+0000", 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303540+0000", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209275 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303556+0000", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.303566+0000", 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6330000000000008e-06 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305547+0000", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019808759999999999 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305580+0000", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3392999999999997e-05 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.312952+0000", 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073721610000000003 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313042+0000", 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0082999999999998e-05 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:595 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'11, mlcod=48'4)", 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.601665632, 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090825420000000007, 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304056+0000", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304057+0000", 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.167e-06 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304261+0000", 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020368799999999999 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304278+0000", 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6963e-05 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304291+0000", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3030000000000001e-05 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308945+0000", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046539190000000003 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.308987+0000", 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.176e-05 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313058+0000", 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040717189999999997 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313139+0000", 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0296000000000005e-05 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:596 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'12, mlcod=48'4)", 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.600940010999999, 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094089699999999991, 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304782+0000", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304783+0000", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0839999999999999e-06 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.304989+0000", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000206512 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305009+0000", 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9230000000000001e-05 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305032+0000", 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3759000000000001e-05 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310178+0000", 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005145658 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.310237+0000", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8950000000000003e-05 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314107+0000", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038697570000000001 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314191+0000", 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4019999999999999e-05 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:597 2.0 e54/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 54'13, mlcod=48'4)", 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.600203039, 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0093279000000000001, 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305519+0000", 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305520+0000", 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.683e-06 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305729+0000", 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020902200000000001 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305763+0000", 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3949000000000003e-05 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.305803+0000", 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9873999999999998e-05 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313198+0000", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073946469999999999 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.313237+0000", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9082999999999998e-05 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314765+0000", 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015280789999999999 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.314847+0000", 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1563000000000005e-05 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:718 2.3 e55/17 2:d9bcccc0:::rbd_data.122ca41dcc05.000000000000000e:head v 55'32, mlcod=55'30)", 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.945799501000003, 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069827379999999996, 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959922+0000", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.959924+0000", 2024-09-21T23:39:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.79e-06 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960641+0000", 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000716988 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960670+0000", 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8756999999999999e-05 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960683+0000", 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.361e-05 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960817+0000", 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343200000000001 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.960875+0000", 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8003e-05 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966828+0000", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005952784 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.966905+0000", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7373999999999995e-05 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:728 2.3 e55/17 2:cb6a5a27:::rbd_data.122ca41dcc05.000000000000000f:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.883621097999999, 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073640579999999997, 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022101+0000", 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022102+0000", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.826e-06 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022814+0000", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071130200000000003 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022842+0000", 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8549999999999999e-05 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022856+0000", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3390000000000001e-05 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.022987+0000", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013153899999999999 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.023066+0000", 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9011999999999995e-05 2024-09-21T23:39:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029371+0000", 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063049730000000002 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:28.029465+0000", 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3466000000000003e-05 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:31 2.0 e75/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 75'89, mlcod=72'87)", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.391224996999998, 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065639640000000003, 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514497+0000", 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514499+0000", 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.877e-06 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514510+0000", 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1663e-05 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514540+0000", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0096000000000001e-05 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514560+0000", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9317000000000002e-05 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514684+0000", 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012411200000000001 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.514721+0000", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7707000000000003e-05 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.520975+0000", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062540119999999998 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.521061+0000", 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5179999999999994e-05 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:41 2.0 e75/37 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head v 75'91, mlcod=72'88)", 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.370441370999998, 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089977249999999998, 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535280+0000", 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535283+0000", 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535304+0000", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0619999999999999e-05 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535351+0000", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7719999999999997e-05 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535373+0000", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1552999999999998e-05 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535531+0000", 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000157885 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.535593+0000", 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2399999999999999e-05 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544205+0000", 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086119019999999994 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.544278+0000", 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2974999999999998e-05 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:43 2.3 e75/17 2:c6346a10:::rbd_data.122ca41dcc05.0000000000000007:head v 75'98, mlcod=64'95)", 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.369183567, 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010489332000000001, 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536538+0000", 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536540+0000", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536561+0000", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0933000000000001e-05 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536599+0000", 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7963000000000002e-05 2024-09-21T23:39:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536618+0000", 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9040000000000001e-05 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536799+0000", 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018089199999999999 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.536869+0000", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0436000000000001e-05 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.546957+0000", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010087631999999999 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547027+0000", 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0363000000000004e-05 2024-09-21T23:39:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:92 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'108, mlcod=76'100)", 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.292586624999998, 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083907579999999999, 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613135+0000", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613136+0000", 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4499999999999996e-07 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613361+0000", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022507399999999999 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613378+0000", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6731999999999998e-05 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613391+0000", 2024-09-21T23:39:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3047000000000001e-05 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615189+0000", 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001797931 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5012000000000003e-05 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621392+0000", 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061686850000000001 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621526+0000", 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013353200000000001 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:93 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'109, mlcod=76'100)", 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.291937100999998, 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077928269999999996, 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613785+0000", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0699999999999996e-07 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613964+0000", 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017884699999999999 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613979+0000", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243000000000001e-05 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.613987+0000", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7900000000000005e-06 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616257+0000", 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022696130000000002 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616287+0000", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0450000000000002e-05 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621532+0000", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052442909999999999 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.621577+0000", 2024-09-21T23:39:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5686000000000003e-05 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:94 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'110, mlcod=76'100)", 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.291168115000001, 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086000920000000002, 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614554+0000", 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614657+0000", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102179 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614668+0000", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1505000000000001e-05 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.614675+0000", 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617112+0000", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024372949999999999 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.617146+0000", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4162999999999998e-05 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623080+0000", 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059341530000000002 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623154+0000", 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3212000000000003e-05 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:95 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'111, mlcod=76'100)", 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.290499027999999, 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079773919999999998, 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615223+0000", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615224+0000", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6599999999999994e-07 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615366+0000", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000142585 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615383+0000", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7244999999999998e-05 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.615394+0000", 2024-09-21T23:39:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0715e-05 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618154+0000", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027597450000000001 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618188+0000", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4217000000000001e-05 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623158+0000", 2024-09-21T23:39:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049695989999999999 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623200+0000", 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2320000000000001e-05 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:96 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'112, mlcod=76'102)", 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.289611662999999, 2024-09-21T23:39:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071258629999999996, 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616110+0000", 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616179+0000", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8514000000000002e-05 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616260+0000", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1724999999999994e-05 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616274+0000", 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3884e-05 2024-09-21T23:39:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616283+0000", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2289999999999992e-06 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618980+0000", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026965980000000001 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.618993+0000", 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2835e-05 2024-09-21T23:39:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623203+0000", 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042096950000000003 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623236+0000", 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3383000000000002e-05 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:97 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'113, mlcod=76'102)", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.288984503000002, 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065371459999999998, 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616737+0000", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616738+0000", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616847+0000", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010869700000000001 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616860+0000", 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145e-05 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.616867+0000", 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2019999999999998e-06 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619479+0000", 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002611642 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.619511+0000", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2246000000000003e-05 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623237+0000", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037264609999999999 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.623274+0000", 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6925999999999998e-05 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:101 2.0 e76/37 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head v 76'117, mlcod=76'106)", 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.285630886, 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066386559999999997, 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620091+0000", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620092+0000", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7199999999999997e-07 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620282+0000", 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019049 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620317+0000", 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4402e-05 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.620331+0000", 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4338000000000001e-05 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622069+0000", 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001738126 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.622100+0000", 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1272999999999997e-05 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626643+0000", 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004543025 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.626729+0000", 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6130000000000004e-05 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5898.0:1 2.1 2:8d833f37:::journal.122ca41dcc05:head [call journal.client_register in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:14.902828+0000", 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.002894049, 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065996630000000004, 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5898", 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/4190215707", 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902828+0000", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902828+0000", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902830+0000", 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.773e-06 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902844+0000", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3933e-05 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902868+0000", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3773000000000001e-05 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.902889+0000", 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.048e-05 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.903062+0000", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017317600000000001 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.903073+0000", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1695999999999999e-05 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.906474+0000", 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003400354 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.906614+0000", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014024299999999999 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.907065+0000", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045053600000000002 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.908302+0000", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001237681 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.909277+0000", 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097501599999999999 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.909363+0000", 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6149999999999993e-05 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:14.909427+0000", 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3851999999999998e-05 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:23.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:23.110 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:23.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:39:23.111 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:23.112 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:39:23.112 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:39:23.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:23.113 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:23.109+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:23.211 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:23.209+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:23.241 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:23.242 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:23.242 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:23.242 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:23.255 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:23.312 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:23.310+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:23.377 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:23.378 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:23.391 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:23.413 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:23.413+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:23.481 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:23.497 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-21T23:39:23.074444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149562275,"num_objects":77,"num_object_clones":20,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":12,"num_read":6716,"num_read_kb":179777,"num_write":1025,"num_write_kb":275338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1015,"ondisk_log_size":1015,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":485356,"kb_used_data":285748,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565745684,"statfs":{"total":579820584960,"available":579323580416,"internally_reserved":0,"allocated":292605952,"data_stored":309942177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-312,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":953,"num_read_kb":2537,"num_write":49,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023216"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":299,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054183+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:19.054183+0000","last_peered":"2024-09-21T23:39:19.054183+0000","last_clean":"2024-09-21T23:39:19.054183+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:19.054183+0000","last_undegraded":"2024-09-21T23:39:19.054183+0000","last_fullsized":"2024-09-21T23:39:19.054183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"76'105","reported_seq":1143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.628009+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:39:19.628009+0000","last_peered":"2024-09-21T23:39:19.628009+0000","last_clean":"2024-09-21T23:39:19.628009+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:19.628009+0000","last_undegraded":"2024-09-21T23:39:19.628009+0000","last_fullsized":"2024-09-21T23:39:19.628009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:26.764339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":20971574,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1121,"num_read_kb":45329,"num_write":86,"num_write_kb":49221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"67'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054189+0000","last_change":"2024-09-21T23:39:02.538313+0000","last_active":"2024-09-21T23:39:19.054189+0000","last_peered":"2024-09-21T23:39:19.054189+0000","last_clean":"2024-09-21T23:39:19.054189+0000","last_became_active":"2024-09-21T23:39:02.532650+0000","last_became_peered":"2024-09-21T23:39:02.532650+0000","last_unstale":"2024-09-21T23:39:19.054189+0000","last_undegraded":"2024-09-21T23:39:19.054189+0000","last_fullsized":"2024-09-21T23:39:19.054189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:22:53.748798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021607010000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'143","reported_seq":728,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.619490+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:39:19.619490+0000","last_peered":"2024-09-21T23:39:19.619490+0000","last_clean":"2024-09-21T23:39:19.619490+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:19.619490+0000","last_undegraded":"2024-09-21T23:39:19.619490+0000","last_fullsized":"2024-09-21T23:39:19.619490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:05.728165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089190,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":488,"num_read_kb":30139,"num_write":136,"num_write_kb":41035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728482+0000","last_change":"2024-09-21T23:38:24.145472+0000","last_active":"2024-09-21T23:39:19.728482+0000","last_peered":"2024-09-21T23:39:19.728482+0000","last_clean":"2024-09-21T23:39:19.728482+0000","last_became_active":"2024-09-21T23:38:22.083188+0000","last_became_peered":"2024-09-21T23:38:22.083188+0000","last_unstale":"2024-09-21T23:39:19.728482+0000","last_undegraded":"2024-09-21T23:39:19.728482+0000","last_fullsized":"2024-09-21T23:39:19.728482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:42.715435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065327500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"83'63","reported_seq":495,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728498+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:39:19.728498+0000","last_peered":"2024-09-21T23:39:19.728498+0000","last_clean":"2024-09-21T23:39:19.728498+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:19.728498+0000","last_undegraded":"2024-09-21T23:39:19.728498+0000","last_fullsized":"2024-09-21T23:39:19.728498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:37.140312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":53,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728426+0000","last_change":"2024-09-21T23:39:13.707817+0000","last_active":"2024-09-21T23:39:19.728426+0000","last_peered":"2024-09-21T23:39:19.728426+0000","last_clean":"2024-09-21T23:39:19.728426+0000","last_became_active":"2024-09-21T23:39:13.707133+0000","last_became_peered":"2024-09-21T23:39:13.707133+0000","last_unstale":"2024-09-21T23:39:19.728426+0000","last_undegraded":"2024-09-21T23:39:19.728426+0000","last_fullsized":"2024-09-21T23:39:19.728426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:30.943757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"75'45","reported_seq":387,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.540810+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:39:19.540810+0000","last_peered":"2024-09-21T23:39:19.540810+0000","last_clean":"2024-09-21T23:39:19.540810+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:19.540810+0000","last_undegraded":"2024-09-21T23:39:19.540810+0000","last_fullsized":"2024-09-21T23:39:19.540810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:16.435855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":20971539,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":192,"num_read_kb":24698,"num_write":43,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"67'2","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054266+0000","last_change":"2024-09-21T23:38:24.108228+0000","last_active":"2024-09-21T23:39:19.054266+0000","last_peered":"2024-09-21T23:39:19.054266+0000","last_clean":"2024-09-21T23:39:19.054266+0000","last_became_active":"2024-09-21T23:38:22.086357+0000","last_became_peered":"2024-09-21T23:38:22.086357+0000","last_unstale":"2024-09-21T23:39:19.054266+0000","last_undegraded":"2024-09-21T23:39:19.054266+0000","last_fullsized":"2024-09-21T23:39:19.054266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:29.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498510000000001,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.531337+0000","last_change":"2024-09-21T23:38:24.115041+0000","last_active":"2024-09-21T23:39:19.531337+0000","last_peered":"2024-09-21T23:39:19.531337+0000","last_clean":"2024-09-21T23:39:19.531337+0000","last_became_active":"2024-09-21T23:38:22.086737+0000","last_became_peered":"2024-09-21T23:38:22.086737+0000","last_unstale":"2024-09-21T23:39:19.531337+0000","last_undegraded":"2024-09-21T23:39:19.531337+0000","last_fullsized":"2024-09-21T23:39:19.531337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:01:41.242218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057152899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"83'6","reported_seq":194,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728759+0000","last_change":"2024-09-21T23:39:13.712876+0000","last_active":"2024-09-21T23:39:19.728759+0000","last_peered":"2024-09-21T23:39:19.728759+0000","last_clean":"2024-09-21T23:39:19.728759+0000","last_became_active":"2024-09-21T23:39:13.710444+0000","last_became_peered":"2024-09-21T23:39:13.710444+0000","last_unstale":"2024-09-21T23:39:19.728759+0000","last_undegraded":"2024-09-21T23:39:19.728759+0000","last_fullsized":"2024-09-21T23:39:19.728759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:18.416845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":5,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"67'1","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728528+0000","last_change":"2024-09-21T23:39:13.708355+0000","last_active":"2024-09-21T23:39:19.728528+0000","last_peered":"2024-09-21T23:39:19.728528+0000","last_clean":"2024-09-21T23:39:19.728528+0000","last_became_active":"2024-09-21T23:39:13.707573+0000","last_became_peered":"2024-09-21T23:39:13.707573+0000","last_unstale":"2024-09-21T23:39:19.728528+0000","last_undegraded":"2024-09-21T23:39:19.728528+0000","last_fullsized":"2024-09-21T23:39:19.728528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:38.767732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078167000000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"83'197","reported_seq":613,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054265+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:39:19.054265+0000","last_peered":"2024-09-21T23:39:19.054265+0000","last_clean":"2024-09-21T23:39:19.054265+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:19.054265+0000","last_undegraded":"2024-09-21T23:39:19.054265+0000","last_fullsized":"2024-09-21T23:39:19.054265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:35:52.224349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":20971612,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":993,"num_read_kb":19600,"num_write":300,"num_write_kb":37083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728582+0000","last_change":"2024-09-21T23:38:24.185684+0000","last_active":"2024-09-21T23:39:19.728582+0000","last_peered":"2024-09-21T23:39:19.728582+0000","last_clean":"2024-09-21T23:39:19.728582+0000","last_became_active":"2024-09-21T23:38:22.085483+0000","last_became_peered":"2024-09-21T23:38:22.085483+0000","last_unstale":"2024-09-21T23:39:19.728582+0000","last_undegraded":"2024-09-21T23:39:19.728582+0000","last_fullsized":"2024-09-21T23:39:19.728582+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:36.524587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064978400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"83'91","reported_seq":359,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.053996+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:39:19.053996+0000","last_peered":"2024-09-21T23:39:19.053996+0000","last_clean":"2024-09-21T23:39:19.053996+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:19.053996+0000","last_undegraded":"2024-09-21T23:39:19.053996+0000","last_fullsized":"2024-09-21T23:39:19.053996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:02:33.022971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":21020748,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":229,"num_read_kb":24679,"num_write":103,"num_write_kb":45142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"67'1","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728755+0000","last_change":"2024-09-21T23:38:24.145476+0000","last_active":"2024-09-21T23:39:19.728755+0000","last_peered":"2024-09-21T23:39:19.728755+0000","last_clean":"2024-09-21T23:39:19.728755+0000","last_became_active":"2024-09-21T23:38:22.084569+0000","last_became_peered":"2024-09-21T23:38:22.084569+0000","last_unstale":"2024-09-21T23:39:19.728755+0000","last_undegraded":"2024-09-21T23:39:19.728755+0000","last_fullsized":"2024-09-21T23:39:19.728755+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:02.205745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060418199999999998,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'239","reported_seq":1420,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.611942+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:39:19.611942+0000","last_peered":"2024-09-21T23:39:19.611942+0000","last_clean":"2024-09-21T23:39:19.611942+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:19.611942+0000","last_undegraded":"2024-09-21T23:39:19.611942+0000","last_fullsized":"2024-09-21T23:39:19.611942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:58:28.988738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":25169958,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":3134,"num_read_kb":32302,"num_write":134,"num_write_kb":47145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":30,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":148971837,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":12,"num_read":6562,"num_read_kb":179647,"num_write":860,"num_write_kb":272916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":285581312,"data_stored":306528890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":889,"ondisk_log_size":889,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607582,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95772,"kb_used_data":60424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94276068,"statfs":{"total":96636764160,"available":96538693632,"internally_reserved":0,"allocated":61874176,"data_stored":71730188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607583,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58856,"kb_used_data":23504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312984,"statfs":{"total":96636764160,"available":96576495616,"internally_reserved":0,"allocated":24068096,"data_stored":23508834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100460,"kb_used_data":65112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271380,"statfs":{"total":96636764160,"available":96533893120,"internally_reserved":0,"allocated":66674688,"data_stored":66091565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607582,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83416,"kb_used_data":52228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94288424,"statfs":{"total":96636764160,"available":96551346176,"internally_reserved":0,"allocated":53481472,"data_stored":63341561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607581,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89552,"kb_used_data":58364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282288,"statfs":{"total":96636764160,"available":96545062912,"internally_reserved":0,"allocated":59764736,"data_stored":59144129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607582,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":26116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":26742784,"data_stored":26125900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25178112,"data_stored":25165916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58781696,"data_stored":58769503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52518912,"data_stored":62971996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65101824,"data_stored":65126606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23089152,"data_stored":23134246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":60911616,"data_stored":71360623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":54,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:23.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:23.514 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:23.514+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:23.561 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 131.329395898, 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.562 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:39:23.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:39:23.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:23.568 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.982090655, 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:39:23.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:39:23.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:39:23.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:39:23.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.982011221999997, 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:23.574 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:39:23.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.981980219, 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.981935661000001, 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:23.582 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:23.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:39:23.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:39:23.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.981900856000003, 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:39:23.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:39:23.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.958287632999998, 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:39:23.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.595 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.958252717000001, 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.596 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.601 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.958220967999999, 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.958181306999997, 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.607 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:39:23.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.958150068000002, 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:39:23.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.618 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.957695690999998, 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.957321507000003, 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:39:23.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.955064098000001, 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.635 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.950720412999999, 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.94455679, 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.641 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.646 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.941326895000003, 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:39:23.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:39:23.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.652 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.940940638999997, 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.931179258999997, 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:39:23.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:39:23.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.930950002000003, 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:23.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:39:23.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:39:23.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:39:23.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:23.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:23.669 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:23.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:23.671 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:23.670+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:23.772 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:23.770+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:23.832 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:23.839 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:23.840 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:23.850 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-21T23:39:23.074444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149562275,"num_objects":77,"num_object_clones":20,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":12,"num_read":6716,"num_read_kb":179777,"num_write":1025,"num_write_kb":275338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1015,"ondisk_log_size":1015,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":485356,"kb_used_data":285748,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565745684,"statfs":{"total":579820584960,"available":579323580416,"internally_reserved":0,"allocated":292605952,"data_stored":309942177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-312,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":953,"num_read_kb":2537,"num_write":49,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023216"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":299,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054183+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:19.054183+0000","last_peered":"2024-09-21T23:39:19.054183+0000","last_clean":"2024-09-21T23:39:19.054183+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:19.054183+0000","last_undegraded":"2024-09-21T23:39:19.054183+0000","last_fullsized":"2024-09-21T23:39:19.054183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"76'105","reported_seq":1143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.628009+0000","last_change":"2024-09-21T23:37:19.282766+0000","last_active":"2024-09-21T23:39:19.628009+0000","last_peered":"2024-09-21T23:39:19.628009+0000","last_clean":"2024-09-21T23:39:19.628009+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:19.628009+0000","last_undegraded":"2024-09-21T23:39:19.628009+0000","last_fullsized":"2024-09-21T23:39:19.628009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:26.764339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087967600000000003,"stat_sum":{"num_bytes":20971574,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1121,"num_read_kb":45329,"num_write":86,"num_write_kb":49221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"67'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054189+0000","last_change":"2024-09-21T23:39:02.538313+0000","last_active":"2024-09-21T23:39:19.054189+0000","last_peered":"2024-09-21T23:39:19.054189+0000","last_clean":"2024-09-21T23:39:19.054189+0000","last_became_active":"2024-09-21T23:39:02.532650+0000","last_became_peered":"2024-09-21T23:39:02.532650+0000","last_unstale":"2024-09-21T23:39:19.054189+0000","last_undegraded":"2024-09-21T23:39:19.054189+0000","last_fullsized":"2024-09-21T23:39:19.054189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:22:53.748798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021607010000000001,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'143","reported_seq":728,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.619490+0000","last_change":"2024-09-21T23:37:55.958967+0000","last_active":"2024-09-21T23:39:19.619490+0000","last_peered":"2024-09-21T23:39:19.619490+0000","last_clean":"2024-09-21T23:39:19.619490+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:19.619490+0000","last_undegraded":"2024-09-21T23:39:19.619490+0000","last_fullsized":"2024-09-21T23:39:19.619490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:05.728165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089190,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":488,"num_read_kb":30139,"num_write":136,"num_write_kb":41035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728482+0000","last_change":"2024-09-21T23:38:24.145472+0000","last_active":"2024-09-21T23:39:19.728482+0000","last_peered":"2024-09-21T23:39:19.728482+0000","last_clean":"2024-09-21T23:39:19.728482+0000","last_became_active":"2024-09-21T23:38:22.083188+0000","last_became_peered":"2024-09-21T23:38:22.083188+0000","last_unstale":"2024-09-21T23:39:19.728482+0000","last_undegraded":"2024-09-21T23:39:19.728482+0000","last_fullsized":"2024-09-21T23:39:19.728482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:42.715435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065327500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"83'63","reported_seq":495,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728498+0000","last_change":"2024-09-21T23:37:19.296725+0000","last_active":"2024-09-21T23:39:19.728498+0000","last_peered":"2024-09-21T23:39:19.728498+0000","last_clean":"2024-09-21T23:39:19.728498+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:19.728498+0000","last_undegraded":"2024-09-21T23:39:19.728498+0000","last_fullsized":"2024-09-21T23:39:19.728498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:37.140312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064161999999999997,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":53,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728426+0000","last_change":"2024-09-21T23:39:13.707817+0000","last_active":"2024-09-21T23:39:19.728426+0000","last_peered":"2024-09-21T23:39:19.728426+0000","last_clean":"2024-09-21T23:39:19.728426+0000","last_became_active":"2024-09-21T23:39:13.707133+0000","last_became_peered":"2024-09-21T23:39:13.707133+0000","last_unstale":"2024-09-21T23:39:19.728426+0000","last_undegraded":"2024-09-21T23:39:19.728426+0000","last_fullsized":"2024-09-21T23:39:19.728426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:22:30.943757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"75'45","reported_seq":387,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.540810+0000","last_change":"2024-09-21T23:37:19.283941+0000","last_active":"2024-09-21T23:39:19.540810+0000","last_peered":"2024-09-21T23:39:19.540810+0000","last_clean":"2024-09-21T23:39:19.540810+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:19.540810+0000","last_undegraded":"2024-09-21T23:39:19.540810+0000","last_fullsized":"2024-09-21T23:39:19.540810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:16.435855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020467409999999999,"stat_sum":{"num_bytes":20971539,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":192,"num_read_kb":24698,"num_write":43,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"67'2","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054266+0000","last_change":"2024-09-21T23:38:24.108228+0000","last_active":"2024-09-21T23:39:19.054266+0000","last_peered":"2024-09-21T23:39:19.054266+0000","last_clean":"2024-09-21T23:39:19.054266+0000","last_became_active":"2024-09-21T23:38:22.086357+0000","last_became_peered":"2024-09-21T23:38:22.086357+0000","last_unstale":"2024-09-21T23:39:19.054266+0000","last_undegraded":"2024-09-21T23:39:19.054266+0000","last_fullsized":"2024-09-21T23:39:19.054266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:29.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498510000000001,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.531337+0000","last_change":"2024-09-21T23:38:24.115041+0000","last_active":"2024-09-21T23:39:19.531337+0000","last_peered":"2024-09-21T23:39:19.531337+0000","last_clean":"2024-09-21T23:39:19.531337+0000","last_became_active":"2024-09-21T23:38:22.086737+0000","last_became_peered":"2024-09-21T23:38:22.086737+0000","last_unstale":"2024-09-21T23:39:19.531337+0000","last_undegraded":"2024-09-21T23:39:19.531337+0000","last_fullsized":"2024-09-21T23:39:19.531337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:01:41.242218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057152899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"83'6","reported_seq":194,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728759+0000","last_change":"2024-09-21T23:39:13.712876+0000","last_active":"2024-09-21T23:39:19.728759+0000","last_peered":"2024-09-21T23:39:19.728759+0000","last_clean":"2024-09-21T23:39:19.728759+0000","last_became_active":"2024-09-21T23:39:13.710444+0000","last_became_peered":"2024-09-21T23:39:13.710444+0000","last_unstale":"2024-09-21T23:39:19.728759+0000","last_undegraded":"2024-09-21T23:39:19.728759+0000","last_fullsized":"2024-09-21T23:39:19.728759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:18.416845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075675500000000001,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":5,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"67'1","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728528+0000","last_change":"2024-09-21T23:39:13.708355+0000","last_active":"2024-09-21T23:39:19.728528+0000","last_peered":"2024-09-21T23:39:19.728528+0000","last_clean":"2024-09-21T23:39:19.728528+0000","last_became_active":"2024-09-21T23:39:13.707573+0000","last_became_peered":"2024-09-21T23:39:13.707573+0000","last_unstale":"2024-09-21T23:39:19.728528+0000","last_undegraded":"2024-09-21T23:39:19.728528+0000","last_fullsized":"2024-09-21T23:39:19.728528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:23:38.767732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078167000000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"83'197","reported_seq":613,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.054265+0000","last_change":"2024-09-21T23:37:19.277632+0000","last_active":"2024-09-21T23:39:19.054265+0000","last_peered":"2024-09-21T23:39:19.054265+0000","last_clean":"2024-09-21T23:39:19.054265+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:19.054265+0000","last_undegraded":"2024-09-21T23:39:19.054265+0000","last_fullsized":"2024-09-21T23:39:19.054265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:35:52.224349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012046870000000001,"stat_sum":{"num_bytes":20971612,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":993,"num_read_kb":19600,"num_write":300,"num_write_kb":37083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728582+0000","last_change":"2024-09-21T23:38:24.185684+0000","last_active":"2024-09-21T23:39:19.728582+0000","last_peered":"2024-09-21T23:39:19.728582+0000","last_clean":"2024-09-21T23:39:19.728582+0000","last_became_active":"2024-09-21T23:38:22.085483+0000","last_became_peered":"2024-09-21T23:38:22.085483+0000","last_unstale":"2024-09-21T23:39:19.728582+0000","last_undegraded":"2024-09-21T23:39:19.728582+0000","last_fullsized":"2024-09-21T23:39:19.728582+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:57:36.524587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064978400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"83'91","reported_seq":359,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.053996+0000","last_change":"2024-09-21T23:37:19.272836+0000","last_active":"2024-09-21T23:39:19.053996+0000","last_peered":"2024-09-21T23:39:19.053996+0000","last_clean":"2024-09-21T23:39:19.053996+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:19.053996+0000","last_undegraded":"2024-09-21T23:39:19.053996+0000","last_fullsized":"2024-09-21T23:39:19.053996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:02:33.022971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010641419999999999,"stat_sum":{"num_bytes":21020748,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":229,"num_read_kb":24679,"num_write":103,"num_write_kb":45142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"67'1","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.728755+0000","last_change":"2024-09-21T23:38:24.145476+0000","last_active":"2024-09-21T23:39:19.728755+0000","last_peered":"2024-09-21T23:39:19.728755+0000","last_clean":"2024-09-21T23:39:19.728755+0000","last_became_active":"2024-09-21T23:38:22.084569+0000","last_became_peered":"2024-09-21T23:38:22.084569+0000","last_unstale":"2024-09-21T23:39:19.728755+0000","last_undegraded":"2024-09-21T23:39:19.728755+0000","last_fullsized":"2024-09-21T23:39:19.728755+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:38:21.054959+0000","last_clean_scrub_stamp":"2024-09-21T23:38:21.054959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:30:02.205745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060418199999999998,"stat_sum":{"num_bytes":35,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"84'239","reported_seq":1420,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-21T23:39:19.611942+0000","last_change":"2024-09-21T23:37:19.277603+0000","last_active":"2024-09-21T23:39:19.611942+0000","last_peered":"2024-09-21T23:39:19.611942+0000","last_clean":"2024-09-21T23:39:19.611942+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:19.611942+0000","last_undegraded":"2024-09-21T23:39:19.611942+0000","last_fullsized":"2024-09-21T23:39:19.611942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:58:28.988738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001122935,"stat_sum":{"num_bytes":25169958,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":3134,"num_read_kb":32302,"num_write":134,"num_write_kb":47145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":30,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":8,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":148971837,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":12,"num_read":6562,"num_read_kb":179647,"num_write":860,"num_write_kb":272916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":285581312,"data_stored":306528890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":889,"ondisk_log_size":889,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607582,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95772,"kb_used_data":60424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94276068,"statfs":{"total":96636764160,"available":96538693632,"internally_reserved":0,"allocated":61874176,"data_stored":71730188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607583,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58856,"kb_used_data":23504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312984,"statfs":{"total":96636764160,"available":96576495616,"internally_reserved":0,"allocated":24068096,"data_stored":23508834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100460,"kb_used_data":65112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271380,"statfs":{"total":96636764160,"available":96533893120,"internally_reserved":0,"allocated":66674688,"data_stored":66091565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607582,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83416,"kb_used_data":52228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94288424,"statfs":{"total":96636764160,"available":96551346176,"internally_reserved":0,"allocated":53481472,"data_stored":63341561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607581,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89552,"kb_used_data":58364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282288,"statfs":{"total":96636764160,"available":96545062912,"internally_reserved":0,"allocated":59764736,"data_stored":59144129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607582,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":26116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":26742784,"data_stored":26125900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25178112,"data_stored":25165916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58781696,"data_stored":58769503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52518912,"data_stored":62971996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65101824,"data_stored":65126606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23089152,"data_stored":23134246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":60911616,"data_stored":71360623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":54,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:23.851 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:39:23.851 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:39:23.851 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:39:23.851 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:39:23.851 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:39:23.852 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:39:23.855 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:23.872 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:23.873+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:23.973 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:23.973+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:23.974 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:23.974 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "" 2024-09-21T23:39:23.974 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:24.014 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:24.014 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:24.014 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:24.028 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:24.091 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:39:24.114 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:24.114+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:24.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:24.174 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:24.174+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:24.198 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.834059723999999, 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:24.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.202 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.822395685000004, 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:39:24.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.810829456, 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:39:24.207 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:39:24.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.211 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.292490741999998, 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:39:24.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.273355158000001, 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:39:24.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:39:24.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.263950838, 2024-09-21T23:39:24.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:39:24.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:39:24.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.618515711999997, 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:39:24.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:39:24.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:39:24.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.586861329000001, 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:39:24.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.583158255000001, 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:24.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.569007575000001, 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:39:24.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:39:24.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.557644316999998, 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:24.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.546249359999997, 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:39:24.246 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:39:24.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.536640620999997, 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:24.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:39:24.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:32 2.7 e75/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 75'154, mlcod=75'151)", 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.681284589000001, 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010506980000000001, 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:24.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516298+0000", 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0040000000000002e-06 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516317+0000", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9060999999999999e-05 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516356+0000", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8831999999999997e-05 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516380+0000", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4139999999999999e-05 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516532+0000", 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015149900000000001 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516573+0000", 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1007999999999998e-05 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526749+0000", 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010175797 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526803+0000", 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639000000000003e-05 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:60 2.7 e76/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 76'156, mlcod=75'152)", 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.628259186000001, 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071394079999999999, 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:24.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999996e-07 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569332+0000", 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0203e-05 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569361+0000", 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8127999999999999e-05 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569390+0000", 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9263000000000001e-05 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569468+0000", 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8051000000000002e-05 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569493+0000", 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5313e-05 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576410+0000", 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069164969999999997 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576461+0000", 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1348999999999999e-05 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:64 2.7 e76/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 76'158, mlcod=75'152)", 2024-09-21T23:39:24.263 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.620568669000001, 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042077199999999999, 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577014+0000", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.612e-06 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577027+0000", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3570999999999999e-05 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577054+0000", 2024-09-21T23:39:24.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6713999999999999e-05 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577070+0000", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577138+0000", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8003999999999995e-05 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577157+0000", 2024-09-21T23:39:24.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9587000000000001e-05 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581148+0000", 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039903500000000001 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581220+0000", 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2174000000000004e-05 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5254.0:37 2.7 e77/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 77'184, mlcod=77'180)", 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.721251556, 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029648180000000001, 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476330+0000", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476454+0000", 2024-09-21T23:39:24.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012381499999999999 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476468+0000", 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3737000000000001e-05 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476477+0000", 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477554+0000", 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010771789999999999 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477568+0000", 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577000000000001e-05 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479264+0000", 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001696637 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479294+0000", 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9686999999999999e-05 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(osd.3.0:553 2.7 e88/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:6 v 88'246, mlcod=86'243)", 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.114503027, 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033057999999999998, 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.272 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083079+0000", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.257e-06 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083093+0000", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3576e-05 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083135+0000", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2710000000000003e-05 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083163+0000", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7875e-05 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083404+0000", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024076499999999999 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083474+0000", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9769000000000002e-05 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.086264+0000", 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027898799999999998 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.086384+0000", 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119968 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(osd.3.0:556 2.7 e88/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:6 v 88'248, mlcod=86'243)", 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.112941542, 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003584603, 2024-09-21T23:39:24.276 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084641+0000", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1799999999999999e-06 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084658+0000", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759e-05 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084688+0000", 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9876999999999999e-05 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084706+0000", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8122e-05 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084934+0000", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000228254 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084990+0000", 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5566000000000003e-05 2024-09-21T23:39:24.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088105+0000", 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031149799999999998 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088224+0000", 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011886499999999999 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5642.0:26 2.7 e88/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 88'249, mlcod=86'243)", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.1090048610000001, 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035632910000000001, 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:24.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088578+0000", 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0600000000000002e-06 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088638+0000", 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9648999999999998e-05 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088677+0000", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8884000000000002e-05 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088696+0000", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9028000000000001e-05 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088825+0000", 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012916800000000001 2024-09-21T23:39:24.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088876+0000", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1252999999999999e-05 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.092032+0000", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031564430000000001 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.092139+0000", 2024-09-21T23:39:24.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010680599999999999 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:24.285 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:24.286 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:24.287 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:24.285+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:24.306 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:24.306 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:24.322 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:24.322 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:24.338 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:24.338 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:24.358 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:24.358 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:24.378 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:24.378 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:24.387 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:24.386+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:24.398 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:24.398 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:24.457 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:24.457 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:24.473 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:24.517 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:24.518+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:24.589 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:24.590+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:24.641 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:24.655 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:24.697 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:24.698+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:24.792 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:24.793+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:24.813 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:24.813 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:24.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:24.892 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:24.894+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:24.957 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:24.957 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:24.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:24.993 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:24.994+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:25.087 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.087 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.094 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:25.094+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:25.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:25.195 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:25.194+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:25.226 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.227 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:25.295 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:25.293+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:25.360 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.360 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:25.416 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:25.417+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:25.497 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:25.498+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:25.536 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:25.597 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:25.598+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:25.673 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.674 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:25.740 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:25.741+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:25.799 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:25.797+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:25.813 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.813 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:25.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:25.900 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:25.898+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:25.990 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:25.990 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:26.001 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:26.002+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:26.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:26.102 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:26.101+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:26.118 INFO:tasks.workunit.client.0.smithi165.stderr: Removing all snapshots: 25% complete... Removing all snapshots: 50% complete... Removing all snapshots: 75% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-21T23:39:26.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:39:26.128 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:26.126+0000 7f8a28ad3640 0 -- 172.21.15.165:0/2827421850 >> [v2:172.21.15.118:6808/1410364236,v1:172.21.15.118:6809/1410364236] conn(0x7f8a04008c60 msgr2=0x7f8a0400b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:39:26.131 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:26.131 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:26.136 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace '' remove testrbdnbd21555 2024-09-21T23:39:26.146 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:26.202 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:26.202+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:26.287 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:26.287 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:26.287 INFO:tasks.workunit.client.0.smithi165.stderr:2024-09-21T23:39:26.286+0000 7f1e2f81e640 0 -- 172.21.15.165:0/265370252 >> [v2:172.21.15.79:6808/3900686792,v1:172.21.15.79:6809/3900686792] conn(0x7f1e08003460 msgr2=0x7f1e08009dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:39:26.303 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:26.301+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:26.305 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:26.312 INFO:tasks.workunit.client.0.smithi165.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2024-09-21T23:39:26.310+0000 7f1e2e5bc640 0 -- 172.21.15.165:0/265370252 >> [v2:172.21.15.79:6816/2564679849,v1:172.21.15.79:6817/2564679849] conn(0x7f1e14015410 msgr2=0x7f1e14002cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:39:26.404 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:26.405+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:26.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:39:26.471 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:26.471 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:26.485 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:26.505 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:26.505+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:26.524 INFO:tasks.workunit.client.0.smithi165.stderr: Removing image: 100% complete...done. 2024-09-21T23:39:26.533 INFO:tasks.workunit.client.0.smithi165.stderr:+ for ns in '' ${NS} 2024-09-21T23:39:26.533 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace ns status testrbdnbd21555 2024-09-21T23:39:26.558 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:26.558 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:26.574 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:26.575 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:26.591 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:26.592 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:26.593 INFO:tasks.workunit.client.0.smithi165.stdout:Watchers: none 2024-09-21T23:39:26.597 INFO:tasks.workunit.client.0.smithi165.stderr:+ for s in 0.5 1 2 4 8 16 32 2024-09-21T23:39:26.597 INFO:tasks.workunit.client.0.smithi165.stderr:+ sleep 0.5 2024-09-21T23:39:26.605 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:26.605+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:26.611 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 134.383951094, 2024-09-21T23:39:26.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.71275571699999996, 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4171", 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/982308143", 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230680+0000", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230682+0000", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230696+0000", 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4037e-05 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230718+0000", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2393000000000001e-05 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230736+0000", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7260000000000001e-05 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230891+0000", 2024-09-21T23:39:26.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000155625 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.230919+0000", 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8096999999999999e-05 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940485+0000", 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.709565742 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.940815+0000", 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000330317 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.941112+0000", 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029696799999999999 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.942487+0000", 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001374763 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943274+0000", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078647700000000005 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943369+0000", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5709000000000001e-05 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:37:12.943435+0000", 2024-09-21T23:39:26.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6019000000000006e-05 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:81 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.036645851000003, 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022927953000000001, 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 81 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577985+0000", 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577986+0000", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3600000000000002e-07 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.577990+0000", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6330000000000004e-06 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578008+0000", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.791e-05 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578017+0000", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5990000000000002e-06 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597440+0000", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019423306000000001 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.597697+0000", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025686699999999998 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.598549+0000", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00085165700000000002 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600650+0000", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021009549999999998 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600690+0000", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0540999999999997e-05 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600809+0000", 2024-09-21T23:39:26.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119076 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.600913+0000", 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010357300000000001 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:83 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.036566418, 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022972756, 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.628 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 83 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578064+0000", 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578065+0000", 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8700000000000005e-07 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578069+0000", 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7799999999999998e-06 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578078+0000", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5119999999999997e-06 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578086+0000", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.554e-06 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601022+0000", 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022935863000000001 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601037+0000", 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5359999999999999e-05 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:84 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.036535415000003, 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023076611, 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.632 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 84 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578095+0000", 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578096+0000", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578100+0000", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.58e-06 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578109+0000", 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5859999999999994e-06 2024-09-21T23:39:26.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578128+0000", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9188999999999999e-05 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601144+0000", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023015977999999999 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601172+0000", 2024-09-21T23:39:26.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7625000000000001e-05 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:85 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.036490856999997, 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023074269000000001, 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.636 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 85 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578140+0000", 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578141+0000", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578145+0000", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.16e-06 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578154+0000", 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7799999999999995e-06 2024-09-21T23:39:26.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578165+0000", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0576e-05 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601202+0000", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023036923000000001 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601214+0000", 2024-09-21T23:39:26.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2147000000000001e-05 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:86 2.5 2.1647a95 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.036456051999998, 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023143759999999999, 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578175+0000", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578196+0000", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0948999999999999e-05 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578207+0000", 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0866999999999999e-05 2024-09-21T23:39:26.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.578218+0000", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0959e-05 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601290+0000", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023071491999999999 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601318+0000", 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8866000000000001e-05 2024-09-21T23:39:26.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:95 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012842829, 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022514352000000001, 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:26.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601788+0000", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601789+0000", 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7300000000000004e-07 2024-09-21T23:39:26.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601793+0000", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.827e-06 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601806+0000", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2993000000000001e-05 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601813+0000", 2024-09-21T23:39:26.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6830000000000008e-06 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620443+0000", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630147 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.620618+0000", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017416600000000001 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621121+0000", 2024-09-21T23:39:26.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050310999999999997 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623976+0000", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028547709999999999 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624199+0000", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022369999999999999 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624258+0000", 2024-09-21T23:39:26.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9110000000000002e-05 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624302+0000", 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3872000000000003e-05 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:96 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012807913000003, 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026639162000000001, 2024-09-21T23:39:26.649 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601823+0000", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601827+0000", 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9140000000000001e-06 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601837+0000", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6020000000000006e-06 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601845+0000", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8369999999999997e-06 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.621779+0000", 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019933915999999999 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.622022+0000", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024314900000000001 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.623007+0000", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00098534699999999992 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625646+0000", 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002639134 2024-09-21T23:39:26.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628268+0000", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002621281 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628346+0000", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8032000000000001e-05 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628462+0000", 2024-09-21T23:39:26.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116284 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:100 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012776164000002, 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026994041999999999, 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601855+0000", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601866+0000", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0219e-05 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601876+0000", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0497e-05 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601884+0000", 2024-09-21T23:39:26.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8189999999999992e-06 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624374+0000", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022490130000000001 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624497+0000", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012327 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.624974+0000", 2024-09-21T23:39:26.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047630799999999999 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628619+0000", 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036448790000000002 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628666+0000", 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7324000000000003e-05 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628754+0000", 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8119000000000001e-05 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628849+0000", 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4513999999999997e-05 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:101 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012736502999999, 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030310301000000001, 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:26.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601894+0000", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601895+0000", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601899+0000", 2024-09-21T23:39:26.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8299999999999998e-06 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601908+0000", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601916+0000", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625725+0000", 2024-09-21T23:39:26.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023809251 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.625843+0000", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011765300000000001 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626322+0000", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00047883099999999998 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628953+0000", 2024-09-21T23:39:26.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026314379999999998 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632033+0000", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003079324 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632113+0000", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0439999999999996e-05 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632204+0000", 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1465000000000006e-05 2024-09-21T23:39:26.665 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:102 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012705263999997, 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033441117999999999, 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.666 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 102 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601925+0000", 2024-09-21T23:39:26.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601926+0000", 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5700000000000002e-07 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601930+0000", 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6799999999999999e-06 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601939+0000", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5030000000000003e-06 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.601947+0000", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.559e-06 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.626962+0000", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025015642000000001 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627085+0000", 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012207599999999999 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.627609+0000", 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00052436599999999996 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632335+0000", 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047265279999999998 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635172+0000", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028362589999999998 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635245+0000", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3697999999999998e-05 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635367+0000", 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012115 2024-09-21T23:39:26.671 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:105 2.5 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.012250887, 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034887524000000003, 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.672 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 105 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602380+0000", 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602381+0000", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.164e-06 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602387+0000", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2299999999999996e-06 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602403+0000", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5379e-05 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602412+0000", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8660000000000001e-06 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629057+0000", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026644522 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629221+0000", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016370800000000001 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629399+0000", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000178426 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637015+0000", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076160070000000002 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637110+0000", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4791999999999996e-05 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637199+0000", 2024-09-21T23:39:26.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9250000000000001e-05 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637267+0000", 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8180000000000001e-05 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:106 2.5 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.011876702999999, 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034811534999999998, 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.677 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 106 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602754+0000", 2024-09-21T23:39:26.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602756+0000", 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8989999999999999e-06 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602766+0000", 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6269999999999995e-06 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602787+0000", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1668999999999999e-05 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.602805+0000", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7442999999999999e-05 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632413+0000", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029608787000000001 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632633+0000", 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021962699999999999 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.632824+0000", 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000190667 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637376+0000", 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045525089999999997 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637397+0000", 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0449999999999999e-05 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637468+0000", 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0986000000000001e-05 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637566+0000", 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7870999999999999e-05 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:108 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.009619293999997, 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035951487999999997, 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 108 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605011+0000", 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605013+0000", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605022+0000", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0499999999999997e-06 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605042+0000", 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0166e-05 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.605055+0000", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2999e-05 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635426+0000", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030370819 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.635567+0000", 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014054700000000001 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.636308+0000", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00074092500000000003 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640775+0000", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044670480000000004 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640793+0000", 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8743e-05 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640861+0000", 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7533999999999994e-05 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640963+0000", 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000102057 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:109 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.688 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.005275609000002, 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031901151000000003, 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 109 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:26.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609355+0000", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609357+0000", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0669999999999999e-06 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609368+0000", 2024-09-21T23:39:26.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.098e-05 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609402+0000", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3458999999999999e-05 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.609423+0000", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1162000000000002e-05 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637657+0000", 2024-09-21T23:39:26.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028233938 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.637853+0000", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000195849 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.638504+0000", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00065148699999999996 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.640980+0000", 2024-09-21T23:39:26.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024760770000000001 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641091+0000", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110554 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641185+0000", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.412e-05 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641256+0000", 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1458000000000006e-05 2024-09-21T23:39:26.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:113 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.999111986000003, 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029368001000000001, 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 113 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615519+0000", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615522+0000", 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615537+0000", 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5017e-05 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615575+0000", 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8518e-05 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.615600+0000", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5162000000000001e-05 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639311+0000", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023710236999999999 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639436+0000", 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125699 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.639952+0000", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051581299999999995 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644580+0000", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046277200000000001 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644716+0000", 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013611999999999999 2024-09-21T23:39:26.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644789+0000", 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2713999999999995e-05 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644887+0000", 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7930999999999995e-05 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:114 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.995882090999999, 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029742641, 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 114 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618749+0000", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618750+0000", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.854e-06 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618761+0000", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0172e-05 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618779+0000", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8349999999999999e-05 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618791+0000", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1678999999999999e-05 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641352+0000", 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022561536 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.641496+0000", 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014338299999999999 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642110+0000", 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061461800000000004 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644696+0000", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002585838 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648289+0000", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035929009999999999 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648373+0000", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4493999999999994e-05 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648491+0000", 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117816 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:116 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.995495835, 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033087867999999999, 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 116 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619135+0000", 2024-09-21T23:39:26.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619138+0000", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619153+0000", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5152e-05 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619185+0000", 2024-09-21T23:39:26.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2249000000000002e-05 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.619208+0000", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2993000000000002e-05 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.642890+0000", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023682209999999999 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643044+0000", 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000154258 2024-09-21T23:39:26.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.643769+0000", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00072445400000000003 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648128+0000", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043591309999999996 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652019+0000", 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003891238 2024-09-21T23:39:26.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652104+0000", 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4302999999999996e-05 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652223+0000", 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119206 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.710 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:118 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.985734454999999, 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.02355353, 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 118 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628896+0000", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628898+0000", 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.063e-06 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628909+0000", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0939e-05 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628940+0000", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0516000000000001e-05 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.628962+0000", 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2036999999999999e-05 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.644960+0000", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015998399 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645099+0000", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000139235 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.645801+0000", 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070186200000000004 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.648267+0000", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024656180000000001 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652240+0000", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003973028 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652333+0000", 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3267e-05 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652450+0000", 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011656599999999999 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4856.0:119 2.5 2:a95e2680:::rbd_object_map.122ca41dcc05:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.985505197999998, 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023517069000000002, 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:26.716 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4856", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/2079880639", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 119 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629125+0000", 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:26.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629127+0000", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.372e-06 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629133+0000", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.348e-06 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629154+0000", 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0733999999999998e-05 2024-09-21T23:39:26.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.629164+0000", 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0411000000000001e-05 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646586+0000", 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017421717 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:26.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.646719+0000", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013263500000000001 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.647257+0000", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053782500000000002 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.651876+0000", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046193579999999996 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:26.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652469+0000", 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059315599999999996 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652557+0000", 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8367000000000003e-05 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.652643+0000", 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5146000000000001e-05 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.722 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:26.722 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:26.722 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:26.723 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:26.723 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:26.724 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:26.724 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:26.725 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:26.722+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:26.725 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:26.824 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:26.825+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:26.894 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:26.894 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:26.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:26.953 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:26.954+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:27.027 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:27.026+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:27.029 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:27.030 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:27.043 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:27.099 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace ns status testrbdnbd21555 2024-09-21T23:39:27.099 INFO:tasks.workunit.client.0.smithi165.stderr:+ grep 'Watchers: none' 2024-09-21T23:39:27.127 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:27.125+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:27.127 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-21T23:39:27.168 INFO:tasks.workunit.client.0.smithi165.stdout:Watchers: none 2024-09-21T23:39:27.168 INFO:tasks.workunit.client.0.smithi165.stderr:+ break 2024-09-21T23:39:27.168 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace ns snap purge testrbdnbd21555 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:612 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'22, mlcod=54'20)", 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.856447828, 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0049766070000000001, 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363521+0000", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.363523+0000", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.587e-06 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364665+0000", 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00114226 2024-09-21T23:39:27.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364693+0000", 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8124999999999999e-05 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364707+0000", 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3783e-05 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364828+0000", 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000120832 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.364868+0000", 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0552000000000001e-05 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368416+0000", 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035482550000000002 2024-09-21T23:39:27.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.368498+0000", 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1212999999999996e-05 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:614 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'24, mlcod=54'22)", 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.844783788999997, 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041447109999999997, 2024-09-21T23:39:27.224 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375185+0000", 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375186+0000", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.31e-06 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375772+0000", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00058589299999999998 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375801+0000", 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8714999999999999e-05 2024-09-21T23:39:27.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375814+0000", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3273e-05 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375927+0000", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112932 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.375966+0000", 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9085999999999998e-05 2024-09-21T23:39:27.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379228+0000", 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032620169999999999 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.379330+0000", 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010148499999999999 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:616 2.7 e54/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 54'26, mlcod=54'24)", 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.833217560000001, 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043108010000000004, 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386751+0000", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.386753+0000", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4959999999999999e-06 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387399+0000", 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00064655099999999998 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387430+0000", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0312e-05 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387446+0000", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6062999999999999e-05 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387594+0000", 2024-09-21T23:39:27.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014846400000000001 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.387629+0000", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5215000000000001e-05 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.390951+0000", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033212879999999999 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.391062+0000", 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000111412 2024-09-21T23:39:27.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:708 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'38, mlcod=55'36)", 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.314878845999999, 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048320680000000001, 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905090+0000", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905091+0000", 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905560+0000", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046858000000000002 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905576+0000", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5795e-05 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905585+0000", 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0650000000000005e-06 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905720+0000", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135155 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.905773+0000", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3281e-05 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909868+0000", 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0040949619999999997 2024-09-21T23:39:27.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.909922+0000", 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4008000000000002e-05 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:712 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'42, mlcod=55'40)", 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.295743262000002, 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029822609999999999, 2024-09-21T23:39:27.237 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924226+0000", 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924227+0000", 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2860000000000001e-06 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924823+0000", 2024-09-21T23:39:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059626700000000004 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924840+0000", 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6439e-05 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924848+0000", 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7919999999999998e-06 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924909+0000", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0387000000000003e-05 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.924930+0000", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1231999999999999e-05 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927100+0000", 2024-09-21T23:39:27.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021704419999999999 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.927208+0000", 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107416 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4754.0:714 2.7 e55/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 55'44, mlcod=55'42)", 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:27.242 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.286338942, 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047953739999999998, 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933630+0000", 2024-09-21T23:39:27.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.933631+0000", 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4e-07 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934230+0000", 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059954799999999999 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934247+0000", 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6617e-05 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934256+0000", 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6549999999999993e-06 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934345+0000", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9577e-05 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.934382+0000", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6829000000000003e-05 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938296+0000", 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039143870000000001 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:27.938425+0000", 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012892099999999999 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:89 2.7 e57/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 57'51, mlcod=57'49)", 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.640903815999998, 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052513819999999998, 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579065+0000", 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579067+0000", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.03e-06 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579088+0000", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1016e-05 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579133+0000", 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4991999999999999e-05 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579155+0000", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2192e-05 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579307+0000", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015156800000000001 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.579366+0000", 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8743999999999999e-05 2024-09-21T23:39:27.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584195+0000", 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048293219999999996 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.584316+0000", 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121518 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:111 2.7 e57/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 57'54, mlcod=57'52)", 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.609249433000002, 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069453750000000002, 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610719+0000", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610721+0000", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.844e-06 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610739+0000", 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8175000000000001e-05 2024-09-21T23:39:27.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610778+0000", 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8108999999999997e-05 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610801+0000", 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3589000000000001e-05 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.610969+0000", 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016819099999999999 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.611027+0000", 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7521e-05 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617542+0000", 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006515234 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.617665+0000", 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122712 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4856.0:112 2.7 e57/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 57'55, mlcod=57'52)", 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.605546359000002, 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004092867, 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.256 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614423+0000", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614425+0000", 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1069999999999999e-06 2024-09-21T23:39:27.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614442+0000", 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7076e-05 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614496+0000", 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4157999999999999e-05 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614515+0000", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9358999999999999e-05 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614662+0000", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014673099999999999 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.614714+0000", 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2506000000000002e-05 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618432+0000", 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0037174299999999999 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:32.618515+0000", 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3499999999999997e-05 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:146 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'87, mlcod=58'85)", 2024-09-21T23:39:27.260 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.591395679000001, 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045939930000000002, 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628573+0000", 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.628575+0000", 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.216e-06 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629917+0000", 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013412999999999999 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629953+0000", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5824999999999997e-05 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.629972+0000", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9856000000000001e-05 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630146+0000", 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017338100000000001 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.630181+0000", 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4808999999999998e-05 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633087+0000", 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029068710000000001 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.633167+0000", 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9734999999999995e-05 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:148 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'89, mlcod=58'87)", 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.580032420999999, 2024-09-21T23:39:27.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039723229999999998, 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639936+0000", 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.639938+0000", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7990000000000001e-06 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640503+0000", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00056432200000000002 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640542+0000", 2024-09-21T23:39:27.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8955000000000002e-05 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640565+0000", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3186000000000001e-05 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640700+0000", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013551899999999999 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.640734+0000", 2024-09-21T23:39:27.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3979000000000001e-05 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643816+0000", 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030816260000000001 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.643909+0000", 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2936999999999995e-05 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.269 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:150 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'91, mlcod=58'89)", 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.568637463999998, 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0034890120000000001, 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651331+0000", 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651333+0000", 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759999999999999e-06 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651756+0000", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042324700000000002 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651786+0000", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0029e-05 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651818+0000", 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1955000000000002e-05 2024-09-21T23:39:27.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651955+0000", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013675300000000001 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.651994+0000", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8902000000000001e-05 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654743+0000", 2024-09-21T23:39:27.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027491410000000001 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.654820+0000", 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7309000000000001e-05 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4816.0:152 2.7 e58/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 58'93, mlcod=58'91)", 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:27.274 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.559028724999997, 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038224639999999998, 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660940+0000", 2024-09-21T23:39:27.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.660943+0000", 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661651+0000", 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00070863999999999996 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661682+0000", 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0378999999999998e-05 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661696+0000", 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661784+0000", 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8607000000000001e-05 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.661818+0000", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3451999999999998e-05 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664658+0000", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028402309999999999 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:36.664763+0000", 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104746 2024-09-21T23:39:27.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:32 2.7 e75/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 75'154, mlcod=75'151)", 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.703672693000001, 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010506980000000001, 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516296+0000", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516298+0000", 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0040000000000002e-06 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516317+0000", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9060999999999999e-05 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516356+0000", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8831999999999997e-05 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516380+0000", 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4139999999999999e-05 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516532+0000", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015149900000000001 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.516573+0000", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1007999999999998e-05 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526749+0000", 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010175797 2024-09-21T23:39:27.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.526803+0000", 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639000000000003e-05 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:60 2.7 e76/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:head v 76'156, mlcod=75'152)", 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.650647289999998, 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0071394079999999999, 2024-09-21T23:39:27.283 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569322+0000", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999996e-07 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569332+0000", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0203e-05 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569361+0000", 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8127999999999999e-05 2024-09-21T23:39:27.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569390+0000", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9263000000000001e-05 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569468+0000", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8051000000000002e-05 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.569493+0000", 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5313e-05 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576410+0000", 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069164969999999997 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576461+0000", 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1348999999999999e-05 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5215.0:64 2.7 e76/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:head v 76'158, mlcod=75'152)", 2024-09-21T23:39:27.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.642956773000002, 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042077199999999999, 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577012+0000", 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577014+0000", 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.612e-06 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577027+0000", 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3570999999999999e-05 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577054+0000", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6713999999999999e-05 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577070+0000", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5707999999999999e-05 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577138+0000", 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8003999999999995e-05 2024-09-21T23:39:27.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.577157+0000", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9587000000000001e-05 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581148+0000", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039903500000000001 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.581220+0000", 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2174000000000004e-05 2024-09-21T23:39:27.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5254.0:37 2.7 e77/17 2:eb787030:::rbd_data.122ca41dcc05.0000000000000001:head v 77'184, mlcod=77'180)", 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.743639659999999, 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029648180000000001, 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476329+0000", 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476330+0000", 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0100000000000001e-06 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476454+0000", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012381499999999999 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476468+0000", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3737000000000001e-05 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.476477+0000", 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477554+0000", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010771789999999999 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.477568+0000", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577000000000001e-05 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479264+0000", 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001696637 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:08.479294+0000", 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9686999999999999e-05 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(osd.3.0:553 2.7 e88/17 2:e94162e2:::rbd_data.122ca41dcc05.000000000000000d:6 v 88'246, mlcod=86'243)", 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1368911309999996, 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033057999999999998, 2024-09-21T23:39:27.296 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083078+0000", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083079+0000", 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.257e-06 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083093+0000", 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3576e-05 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083135+0000", 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2710000000000003e-05 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083163+0000", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7875e-05 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083404+0000", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024076499999999999 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.083474+0000", 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9769000000000002e-05 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.086264+0000", 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027898799999999998 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.086384+0000", 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000119968 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(osd.3.0:556 2.7 e88/17 2:e99f52b4:::rbd_data.122ca41dcc05.000000000000000c:6 v 88'248, mlcod=86'243)", 2024-09-21T23:39:27.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1353296459999997, 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003584603, 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084639+0000", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084641+0000", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1799999999999999e-06 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084658+0000", 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6759e-05 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084688+0000", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9876999999999999e-05 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084706+0000", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8122e-05 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084934+0000", 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000228254 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.084990+0000", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5566000000000003e-05 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088105+0000", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031149799999999998 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088224+0000", 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011886499999999999 2024-09-21T23:39:27.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.5642.0:26 2.7 e88/17 2:e9b20237:::rbd_header.122ca41dcc05:head v 88'249, mlcod=86'243)", 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1313929649999999, 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035632910000000001, 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088576+0000", 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088578+0000", 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0600000000000002e-06 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088638+0000", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9648999999999998e-05 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088677+0000", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8884000000000002e-05 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088696+0000", 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9028000000000001e-05 2024-09-21T23:39:27.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088825+0000", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012916800000000001 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.088876+0000", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1252999999999999e-05 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.092032+0000", 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031564430000000001 2024-09-21T23:39:27.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:23.092139+0000", 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010680599999999999 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.309 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:27.311 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:27.311 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-21T23:39:27.310+0000 7fa20331e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25325) UID: 0 2024-09-21T23:39:27.412 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:27.413+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:27.480 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:27.480 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:27.480 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-21T23:39:27.480 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:27.494 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:27.541 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:27.542+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:27.614 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:27.613+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:27.619 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-21T23:39:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-21T23:39:27.619 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-21T23:39:27.619 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:27.633 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:27.715 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:27.717+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:27.803 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-21T23:39:27.803 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-21T23:39:27.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:33 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 67.706152388999996, 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.035844834999999999, 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 33 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095552+0000", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.095556+0000", 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4429999999999999e-06 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106870+0000", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011314164999999999 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106910+0000", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0441000000000001e-05 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.106938+0000", 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7871000000000001e-05 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107080+0000", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014204300000000001 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107332+0000", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000251962 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.107601+0000", 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026886899999999998 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.125099+0000", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017498514 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131162+0000", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060630299999999996 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131292+0000", 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012904799999999999 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.131397+0000", 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105449 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:37 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 67.692074841999997, 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039313338000000003, 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 37 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109630+0000", 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.109634+0000", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2529999999999998e-06 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118566+0000", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089318990000000001 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118612+0000", 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5711000000000003e-05 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118639+0000", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7132999999999999e-05 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118837+0000", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019820300000000001 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119255+0000", 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041791899999999999 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.119643+0000", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000388038 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148615+0000", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028971696000000002 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148738+0000", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123693 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148852+0000", 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114226 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.148943+0000", 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0567000000000002e-05 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:39 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 67.682983031000006, 2024-09-21T23:39:27.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.034779400000000002, 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 39 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118722+0000", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.118724+0000", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2000000000000001e-06 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128187+0000", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094636530000000007 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128241+0000", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.401e-05 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.128275+0000", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3472000000000001e-05 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138178+0000", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099029640000000002 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138454+0000", 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000275887 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.138706+0000", 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025269399999999998 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.149737+0000", 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011031005 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153367+0000", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036292310000000001 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153440+0000", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2897000000000004e-05 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.153501+0000", 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1387e-05 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4652.0:41 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,writefull 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 67.660935934999998, 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018504598000000001, 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4652", 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/1457535974", 2024-09-21T23:39:27.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 41 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140769+0000", 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.140772+0000", 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9579999999999999e-06 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145507+0000", 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047352690000000003 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145539+0000", 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1736000000000002e-05 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145565+0000", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5999e-05 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.145707+0000", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142879 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146006+0000", 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029843900000000001 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.146347+0000", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034159400000000002 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158084+0000", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011736052 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.158766+0000", 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00068291399999999996 2024-09-21T23:39:27.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159078+0000", 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031178199999999998 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:20.159273+0000", 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000194976 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:27.825 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.557560602999999, 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023342715999999999, 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:27.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244144+0000", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244145+0000", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2529999999999999e-06 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244154+0000", 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-21T23:39:27.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244174+0000", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9669000000000001e-05 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244188+0000", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4117000000000001e-05 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257523+0000", 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013335326999999999 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257754+0000", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023101099999999999 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.267487+0000", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0097321930000000001 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:18 2.0 2:12ebe7a1:::rbd_data.122ca41dcc05.0000000000000006:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:27.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.557431465000001, 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022349779, 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244273+0000", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244275+0000", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.533e-06 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244282+0000", 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6170000000000001e-06 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244304+0000", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1712000000000002e-05 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.244320+0000", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6036e-05 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.256814+0000", 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012493894 2024-09-21T23:39:27.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.257350+0000", 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053653499999999999 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.266623+0000", 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092724520000000005 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4654.0:17 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [sparse-read 0~4194304 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.525627047999997, 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019658156, 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4654", 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/320471779", 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276077+0000", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276078+0000", 2024-09-21T23:39:27.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8299999999999995e-07 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276084+0000", 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1930000000000001e-06 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276108+0000", 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4493000000000001e-05 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.276118+0000", 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7119999999999995e-06 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289317+0000", 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013198939 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.289571+0000", 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025424699999999998 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:38:28.295736+0000", 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0061646890000000001 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:34 2.2 2:5e1fb7a9:::rbd_data.122ca41dcc05.0000000000000003:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.282837936, 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.023817232000000001, 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 34 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518867+0000", 2024-09-21T23:39:27.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518869+0000", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518884+0000", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4742e-05 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518940+0000", 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5745999999999998e-05 2024-09-21T23:39:27.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.518996+0000", 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5943999999999998e-05 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519161+0000", 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016499700000000001 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519412+0000", 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025155899999999999 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519930+0000", 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051799400000000003 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.537789+0000", 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017859223 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542572+0000", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004782171 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542635+0000", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3083999999999994e-05 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542684+0000", 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9135999999999998e-05 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.843 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:36 2.2 2:593b56e4:::rbd_data.122ca41dcc05.0000000000000005:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.282644358999999, 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024223537999999999, 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 36 2024-09-21T23:39:27.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519060+0000", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519061+0000", 2024-09-21T23:39:27.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.283e-06 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519069+0000", 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7190000000000001e-06 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519094+0000", 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4904000000000001e-05 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.519111+0000", 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7251000000000001e-05 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529102+0000", 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099906300000000003 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529230+0000", 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012755500000000001 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.529483+0000", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025336899999999998 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.539134+0000", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096515980000000008 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.542876+0000", 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003741439 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543167+0000", 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029152799999999999 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.543284+0000", 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011626200000000001 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:59 2.3 2:c0268623:::rbd_data.122ca41dcc05.000000000000000b:head [delete] snapc 6=[6,5,3] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.233546570000001, 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018089152000000001, 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 59 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568158+0000", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568160+0000", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9589999999999998e-06 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568169+0000", 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0129999999999999e-06 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568192+0000", 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.266e-05 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568207+0000", 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5804e-05 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568324+0000", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116344 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op must wait for map", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.568338+0000", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4154e-05 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571148+0000", 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028103519999999999 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571230+0000", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-05 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.571406+0000", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017574999999999999 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.576139+0000", 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047326759999999999 2024-09-21T23:39:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586043+0000", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099038500000000005 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586151+0000", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108323 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.586247+0000", 2024-09-21T23:39:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6277e-05 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:92 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2125824~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193723220999999, 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015425227, 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 92 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607981+0000", 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.607982+0000", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4900000000000001e-07 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608111+0000", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012953800000000001 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608118+0000", 2024-09-21T23:39:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5180000000000001e-06 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608121+0000", 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4989999999999999e-06 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614540+0000", 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064183850000000004 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614587+0000", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7521000000000001e-05 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.614665+0000", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7497999999999996e-05 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617185+0000", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025206109999999999 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623310+0000", 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006124694 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623355+0000", 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4842000000000002e-05 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623407+0000", 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1771999999999998e-05 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:93 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2256896~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193577389000001, 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015447901, 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 93 2024-09-21T23:39:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608127+0000", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.41e-07 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608256+0000", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128291 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608264+0000", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7740000000000004e-06 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608269+0000", 2024-09-21T23:39:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2279999999999998e-06 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615063+0000", 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067938929999999996 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615159+0000", 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6428999999999994e-05 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615288+0000", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012889800000000001 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617455+0000", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021670169999999998 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623474+0000", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060186069999999996 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623520+0000", 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6347999999999997e-05 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.623575+0000", 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5074999999999997e-05 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:94 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2387968~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193424568000001, 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017672687999999999, 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 94 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608280+0000", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608281+0000", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7800000000000001e-07 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608423+0000", 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000142819 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608433+0000", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.38e-06 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608440+0000", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615922+0000", 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074819919999999998 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.615968+0000", 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6622000000000002e-05 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616051+0000", 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2434000000000004e-05 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617461+0000", 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014100060000000001 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625814+0000", 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083532390000000002 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625896+0000", 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1805999999999996e-05 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625953+0000", 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6932000000000003e-05 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:95 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2519040~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193253647999999, 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017624719, 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 95 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608451+0000", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608452+0000", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608524+0000", 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2565999999999996e-05 2024-09-21T23:39:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608532+0000", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2879999999999993e-06 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608539+0000", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8040000000000004e-06 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616439+0000", 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007900041 2024-09-21T23:39:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616540+0000", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010035999999999999 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.616676+0000", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0001362 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619529+0000", 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028531250000000002 2024-09-21T23:39:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.625999+0000", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064702960000000004 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626029+0000", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9702000000000001e-05 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626076+0000", 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6656000000000002e-05 2024-09-21T23:39:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:96 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2650112~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193151659000002, 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017659136999999998, 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 96 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608553+0000", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608620+0000", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6256999999999999e-05 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608634+0000", 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608640+0000", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1659999999999999e-06 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617495+0000", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088556810000000007 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617544+0000", 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8162999999999999e-05 2024-09-21T23:39:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.617618+0000", 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4759999999999996e-05 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619535+0000", 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001916374 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626121+0000", 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0065866299999999996 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626166+0000", 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4202999999999997e-05 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626212+0000", 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6434000000000003e-05 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:97 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2781184~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.193054463999999, 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017675217999999999, 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 97 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608650+0000", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608651+0000", 2024-09-21T23:39:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9299999999999998e-07 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608730+0000", 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9598000000000006e-05 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608739+0000", 2024-09-21T23:39:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0119999999999994e-06 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608746+0000", 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2349999999999996e-06 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618001+0000", 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092555099999999998 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618092+0000", 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0552999999999999e-05 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618235+0000", 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014355499999999999 2024-09-21T23:39:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620378+0000", 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021429750000000001 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626257+0000", 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0058784980000000002 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626283+0000", 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.601e-05 2024-09-21T23:39:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626325+0000", 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2679000000000003e-05 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:98 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 2912256~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.192948908999998, 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017700770000000001, 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 98 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608756+0000", 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608873+0000", 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117304 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608887+0000", 2024-09-21T23:39:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3386e-05 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608894+0000", 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618777+0000", 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0098831249999999995 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618825+0000", 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8563000000000002e-05 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.618902+0000", 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6130000000000005e-05 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621131+0000", 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022290230000000001 2024-09-21T23:39:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626371+0000", 2024-09-21T23:39:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052399329999999996 2024-09-21T23:39:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626412+0000", 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1844999999999997e-05 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626456+0000", 2024-09-21T23:39:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4013e-05 2024-09-21T23:39:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:99 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3043328~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.192799246, 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017665733999999999, 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 99 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608905+0000", 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608906+0000", 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.608993+0000", 2024-09-21T23:39:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6874999999999997e-05 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609004+0000", 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0849999999999999e-05 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609010+0000", 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619684+0000", 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010673867 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619760+0000", 2024-09-21T23:39:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6249999999999997e-05 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.619880+0000", 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011974200000000001 2024-09-21T23:39:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622034+0000", 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021542639999999999 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626501+0000", 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044669829999999999 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626528+0000", 2024-09-21T23:39:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626571+0000", 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3123e-05 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:100 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3174400~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.192650944, 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01765369, 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 100 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609054+0000", 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8299999999999996e-07 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609143+0000", 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9022000000000002e-05 2024-09-21T23:39:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609153+0000", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8160000000000005e-06 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609161+0000", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3420000000000008e-06 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620487+0000", 2024-09-21T23:39:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011325052 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620536+0000", 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654999999999998e-05 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.620615+0000", 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8403000000000001e-05 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622168+0000", 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015539270000000001 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626619+0000", 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044508730000000002 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626662+0000", 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2175000000000002e-05 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.626707+0000", 2024-09-21T23:39:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5741999999999997e-05 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.5215.0:101 2.0 2:1bd14456:::rbd_data.122ca41dcc05.0000000000000000:head [write 3305472~131072] snapc 6=[6,5,3] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.192530244, 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020329684000000001, 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "continuous": false, 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-21T23:39:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.5215", 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.165:0/3546672560", 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 101 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-21T23:39:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609174+0000", 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-21T23:39:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609175+0000", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2200000000000004e-07 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609264+0000", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8688000000000002e-05 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609273+0000", 2024-09-21T23:39:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0380000000000005e-06 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.609277+0000", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.036e-06 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621290+0000", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012013429000000001 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621385+0000", 2024-09-21T23:39:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5260000000000006e-05 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.621524+0000", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013853100000000001 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.622385+0000", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00086160000000000002 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629426+0000", 2024-09-21T23:39:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.007040129 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629455+0000", 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9274000000000001e-05 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-21T23:39:05.629504+0000", 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9076999999999998e-05 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-21T23:39:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-21T23:39:27.935 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-21T23:39:27.936 INFO:tasks.workunit.client.0.smithi165.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-09-21T23:39:27.936 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd -p rbd --namespace ns remove testrbdnbd21555 2024-09-21T23:39:27.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:39:27.937 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-21T23:39:27.933+0000 7fa4e07b7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25269) UID: 0 2024-09-21T23:39:28.037 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:28.037+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:28.046 INFO:tasks.workunit.client.0.smithi165.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete...2024-09-21T23:39:28.041+0000 7f7202879640 0 -- 172.21.15.165:0/3191828155 >> [v2:172.21.15.118:6816/466572374,v1:172.21.15.118:6817/466572374] conn(0x555993204720 msgr2=0x555993206bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-21T23:39:28.061 INFO:tasks.workunit.client.0.smithi165.stderr: Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2024-09-21T23:39:28.057+0000 7f7203adb640 0 -- 172.21.15.165:0/3191828155 >> [v2:172.21.15.79:6816/2564679849,v1:172.21.15.79:6817/2564679849] conn(0x7f71e4004560 msgr2=0x7f71e40058a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-21T23:39:28.079 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:28.079 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:28.079 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:28.079 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:28.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:39:28.096 INFO:tasks.workunit.client.0.smithi165.stderr: Removing image: 100% complete...done. 2024-09-21T23:39:28.105 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd namespace remove rbd/ns 2024-09-21T23:39:28.137 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:28.138+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:28.179 INFO:tasks.workunit.client.0.smithi165.stderr:+ rbd config global rm global rbd_default_pool 2024-09-21T23:39:28.223 INFO:tasks.workunit.client.0.smithi165.stderr:+ ceph osd pool delete new_default_pool21555 new_default_pool21555 --yes-i-really-really-mean-it 2024-09-21T23:39:28.238 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:28.238+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:28.257 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:28.257 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:28.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:39:28.338 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:28.338+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:131 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 134.76782508900001, 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004924075, 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628262+0000", 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628264+0000", 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4270000000000001e-06 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628367+0000", 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103372 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628392+0000", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5117e-05 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628409+0000", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6583e-05 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628507+0000", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8536000000000007e-05 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.628544+0000", 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7005999999999998e-05 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633126+0000", 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045819509999999999 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633186+0000", 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0083000000000001e-05 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4171.0:137 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'87)", 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 134.763773007, 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054209039999999998, 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632314+0000", 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632315+0000", 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9700000000000005e-07 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632395+0000", 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9351999999999995e-05 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632437+0000", 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2519999999999999e-05 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.632452+0000", 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5183e-05 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633392+0000", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093971300000000001 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.633427+0000", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5450000000000001e-05 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637670+0000", 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042424969999999996 2024-09-21T23:39:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:37:13.637735+0000", 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5291999999999998e-05 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:667 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'27, mlcod=48'25)", 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.718542691000003, 2024-09-21T23:39:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049675520000000001, 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677545+0000", 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.677547+0000", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678017+0000", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047031599999999997 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678047+0000", 2024-09-21T23:39:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678061+0000", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4283e-05 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678203+0000", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141509 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.678269+0000", 2024-09-21T23:39:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6396000000000006e-05 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682412+0000", 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041429919999999999 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.682512+0000", 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010009300000000001 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4754.0:668 2.5 e54/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:head v 54'28, mlcod=54'26)", 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.711162563999999, 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057114210000000004, 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684925+0000", 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.684927+0000", 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686200+0000", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012735940000000001 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686230+0000", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9597000000000001e-05 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686244+0000", 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.394e-05 2024-09-21T23:39:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686365+0000", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121086 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.686420+0000", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5019000000000003e-05 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690564+0000", 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004143969 2024-09-21T23:39:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:27.690636+0000", 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2316000000000004e-05 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:119 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'106, mlcod=58'104)", 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.920080398000003, 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061536810000000003, 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476007+0000", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476009+0000", 2024-09-21T23:39:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476739+0000", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073007499999999995 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476768+0000", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8836999999999999e-05 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476781+0000", 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3689000000000001e-05 2024-09-21T23:39:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476920+0000", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138689 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.476955+0000", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5262999999999998e-05 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482074+0000", 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051191279999999997 2024-09-21T23:39:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.482161+0000", 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6169999999999997e-05 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:123 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'110, mlcod=58'108)", 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.897076194, 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006306089, 2024-09-21T23:39:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499011+0000", 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499013+0000", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5460000000000001e-06 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499911+0000", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089809700000000004 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499935+0000", 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4527000000000001e-05 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.499948+0000", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2653000000000001e-05 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500064+0000", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011605200000000001 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.500120+0000", 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6032999999999997e-05 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505234+0000", 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051137489999999999 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.505317+0000", 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3431999999999997e-05 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4816.0:125 2.5 e58/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 58'112, mlcod=58'110)", 2024-09-21T23:39:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.883238112000001, 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050597000000000003, 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512849+0000", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.512880+0000", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0397000000000001e-05 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513583+0000", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070352200000000004 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513612+0000", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9312e-05 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513626+0000", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3553999999999999e-05 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513753+0000", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012745500000000001 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.513813+0000", 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9370000000000002e-05 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517706+0000", 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038933639999999999 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:38:36.517909+0000", 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000202726 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:25 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'135, mlcod=75'133)", 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.889135895999999, 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005751763, 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506951+0000", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506953+0000", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506965+0000", 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.184e-05 2024-09-21T23:39:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.506993+0000", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8028999999999999e-05 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507006+0000", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3417e-05 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507142+0000", 2024-09-21T23:39:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013520200000000001 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507195+0000", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3117e-05 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512636+0000", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054415619999999996 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512703+0000", 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6863e-05 2024-09-21T23:39:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:26 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'136, mlcod=75'133)", 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.888446282, 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051681380000000001, 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507641+0000", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507643+0000", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.773e-06 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507658+0000", 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5136999999999999e-05 2024-09-21T23:39:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507687+0000", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9306999999999999e-05 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.507701+0000", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3376e-05 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508223+0000", 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052228599999999995 2024-09-21T23:39:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508266+0000", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3037000000000001e-05 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512707+0000", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044412159999999996 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.512809+0000", 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102006 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:28 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'138, mlcod=75'133)", 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.887200710999998, 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068616110000000001, 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508887+0000", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508888+0000", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.753e-06 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508902+0000", 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3543999999999999e-05 2024-09-21T23:39:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508928+0000", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5809e-05 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.508944+0000", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6390000000000001e-05 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510106+0000", 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001162185 2024-09-21T23:39:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510155+0000", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8912000000000002e-05 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515650+0000", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054947529999999998 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515748+0000", 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8264999999999995e-05 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:29 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'139, mlcod=75'133)", 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.886578452999998, 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063237359999999999, 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509509+0000", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509511+0000", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.747e-06 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509528+0000", 2024-09-21T23:39:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7047000000000001e-05 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509559+0000", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1463e-05 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.509571+0000", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2047e-05 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510923+0000", 2024-09-21T23:39:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001351882 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510973+0000", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9767e-05 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515755+0000", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00478269 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515833+0000", 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7093000000000002e-05 2024-09-21T23:39:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:30 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'140, mlcod=75'133)", 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.885905244, 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057255370000000002, 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510182+0000", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510184+0000", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9360000000000002e-06 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510199+0000", 2024-09-21T23:39:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4564e-05 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510225+0000", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6633000000000001e-05 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.510250+0000", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4323000000000001e-05 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511937+0000", 2024-09-21T23:39:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016875010000000001 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.511974+0000", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7425999999999997e-05 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515836+0000", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038614370000000001 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.515908+0000", 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1717000000000005e-05 2024-09-21T23:39:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:47 2.5 e75/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 75'149, mlcod=75'144)", 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.850816003999999, 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067526640000000002, 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545271+0000", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545272+0000", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6099999999999999e-07 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545279+0000", 2024-09-21T23:39:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4899999999999997e-06 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545293+0000", 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3889e-05 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545301+0000", 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1980000000000001e-06 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545364+0000", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2890000000000003e-05 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.545383+0000", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9074999999999999e-05 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.551967+0000", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065840680000000002 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552024+0000", 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7192999999999999e-05 2024-09-21T23:39:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:48 2.5 e76/17 2:a95e2680:::rbd_object_map.122ca41dcc05:head v 76'150, mlcod=75'147)", 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.849043010999999, 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050129090000000003, 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547044+0000", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547046+0000", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2470000000000001e-06 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547053+0000", 2024-09-21T23:39:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6029999999999999e-06 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547069+0000", 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5863e-05 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547077+0000", 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4239999999999993e-06 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547146+0000", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8106000000000001e-05 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.547170+0000", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4871999999999998e-05 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552029+0000", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048586669999999997 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.552057+0000", 2024-09-21T23:39:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8127e-05 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5215.0:55 2.5 e76/17 2:a331d84f:::rbd_data.122ca41dcc05.0000000000000009:head v 76'159, mlcod=76'153)", 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.838311268999998, 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065210290000000002, 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557776+0000", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4200000000000002e-07 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557781+0000", 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.096e-06 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557790+0000", 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1499999999999999e-06 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557794+0000", 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3749999999999996e-06 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557832+0000", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8475e-05 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.557845+0000", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2607e-05 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564252+0000", 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064069690000000002 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:05.564297+0000", 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5015000000000002e-05 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5868.0:7 2.4 2:2ff85688:::journal_data.2.122ca41dcc05.0:head [read 0~78 [fadvise_dontneed]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.241709066, 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55823445699999996, 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-21T23:39:28.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5868", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/1315715008", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154378+0000", 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154380+0000", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154386+0000", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5470000000000004e-06 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154397+0000", 2024-09-21T23:39:28.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1581e-05 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154409+0000", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.132e-05 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154486+0000", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7667e-05 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.154497+0000", 2024-09-21T23:39:28.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1073000000000001e-05 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.710968+0000", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.55647108099999998 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.711233+0000", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026443799999999998 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:13.712613+0000", 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00137992 2024-09-21T23:39:28.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:563 2.5 e89/17 2:ab864e6e:::rbd_data.122ca41dcc05.0000000000000008:6 v 89'206, mlcod=88'200)", 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:24.098269+0000", 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2978180149999998, 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070787530000000001, 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098269+0000", 2024-09-21T23:39:28.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098269+0000", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098271+0000", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5799999999999999e-06 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098283+0000", 2024-09-21T23:39:28.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2353e-05 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098309+0000", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5984999999999999e-05 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.098337+0000", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7579000000000001e-05 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.099195+0000", 2024-09-21T23:39:28.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085844100000000004 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.099228+0000", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3005e-05 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.105295+0000", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060666549999999998 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:24.105348+0000", 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3155000000000002e-05 2024-09-21T23:39:28.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5659.0:82 2.5 e93/17 2:a6381379:ns::rbd_data.1232433620e5.0000000000000001:head v 93'225, mlcod=91'223)", 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:28.049108+0000", 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.34697966899999999, 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083742290000000004, 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049108+0000", 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049108+0000", 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049109+0000", 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1000000000000001e-06 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049118+0000", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049133+0000", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5429000000000001e-05 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049143+0000", 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6770000000000007e-06 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049241+0000", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7922000000000002e-05 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.049263+0000", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2560000000000001e-05 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.057437+0000", 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081733780000000002 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.057482+0000", 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5139999999999998e-05 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5659.0:84 2.4 2:3caca547:ns::rbd_data.1232433620e5.0000000000000003:head [delete] snapc 4=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e93)", 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:28.050060+0000", 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.346027168, 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0074808849999999996, 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5659", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.165:0/3191828155", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050060+0000", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050060+0000", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050063+0000", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6240000000000002e-06 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050075+0000", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.269e-05 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050118+0000", 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2905999999999999e-05 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050142+0000", 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3839999999999999e-05 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050339+0000", 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019672799999999999 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.050749+0000", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040987700000000001 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.051087+0000", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033800099999999999 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.054021+0000", 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029339420000000001 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.057155+0000", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031342750000000002 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.057314+0000", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158512 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.057541+0000", 2024-09-21T23:39:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022749000000000001 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.5659.0:83 2.5 e93/17 2:bb19007f:ns::rbd_data.1232433620e5.0000000000000002:head v 93'226, mlcod=91'223)", 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-21T23:39:28.053121+0000", 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.34296580999999998, 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050158599999999996, 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053121+0000", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053121+0000", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053122+0000", 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.16e-07 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053132+0000", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0064000000000001e-05 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053143+0000", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0991e-05 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.053151+0000", 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6750000000000002e-06 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.054596+0000", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001445069 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.054653+0000", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6366000000000002e-05 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.058042+0000", 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033894400000000001 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-21T23:39:28.058137+0000", 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5339000000000003e-05 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:28.483 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:28.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:39:28.485 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:28.485+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:28.585 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:28.585+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:28.654 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:28.654 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-21T23:39:28.654 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:28.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:39:28.712 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:28.713+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:28.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:39:28.800 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:28.801+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:28.836 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-21T23:39:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-21T23:39:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-21T23:39:28.836 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:28.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:39:28.894 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-21T23:39:28.895 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:39:28.895 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:28.893+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:28.911 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-21T23:39:28.911 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:39:28.927 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-21T23:39:28.928 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:39:28.948 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-21T23:39:28.948 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:39:28.969 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-21T23:39:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:39:28.987 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:28.985+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:28.989 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:39:28.989 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:39:28.989 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:39:28.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:39:28.990 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-21T23:39:28.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:39:29.019 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.019 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:39:29.088 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:29.086+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:29.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:39:29.196 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:29.197+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:29.198 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.199 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:39:29.222 INFO:tasks.workunit.client.0.smithi165.stderr:pool 'new_default_pool21555' does not exist 2024-09-21T23:39:29.240 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T23:39:29.240 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T23:39:29.293 INFO:tasks.workunit:Stopping ['rbd/rbd-nbd.sh'] on client.0... 2024-09-21T23:39:29.293 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T23:39:29.304 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:29.305+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:29.318 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:29.318 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 94, 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "66973394-6d88-4f14-bf11-7ccd7c9cf4a7", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-21T23:36:58.086363+0000", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-21T23:39:29.126368+0000", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-21T23:37:09.181667+0000", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-21T23:36:59.825771+0000", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5801984, 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-21T23:39:29.319 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 6, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "squid", 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-21T23:39:29.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:11.018666+0000", 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:39:29.321 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:39:29.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "16", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:39:29.323 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:39:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:39:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2024-09-21T23:39:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.92772173881530762, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-21T23:37:15.320283+0000", 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-21T23:39:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "is_stretch_pool": false, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-21T23:39:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "93", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:39:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 7, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 93, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:39:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:39:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "score_type": "Fair distribution", 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.25, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.25, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2.25, 2024-09-21T23:39:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2.25, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.92772173881530762, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "638e3e48-1763-4ec9-8c06-634e0b3b793b", 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 80, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 915397992 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/915397992", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/915397992", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/915397992", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/915397992", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "02967526-d5c5-4f68-8e25-343311dbaec4", 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.695556640625, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3900686792 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/3900686792", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/3900686792", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/3900686792", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/3900686792", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "4f770ac1-d062-4934-81d5-7f980b3830a1", 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.8707733154296875, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 80, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2564679849 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/2564679849", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/2564679849", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6823/2564679849", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/2564679849", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "90cc3313-3845-48d0-839d-8fea4c52bfde", 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 74, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6808", 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6809", 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6810", 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6811", 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6814", 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6815", 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6812", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6813", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1410364236 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6809/1410364236", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6811/1410364236", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6815/1410364236", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6813/1410364236", 2024-09-21T23:39:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "42b8e641-af88-4d41-bc5f-a14020f205ce", 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6800", 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6801", 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6802", 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6803", 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6806", 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6807", 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6804", 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6805", 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1572346429 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6801/1572346429", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6803/1572346429", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6807/1572346429", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6805/1572346429", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9", 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-21T23:39:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 12, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 49, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6816", 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6817", 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6818", 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6819", 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6822", 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6823", 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6820", 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.118:6821", 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 466572374 2024-09-21T23:39:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.118:6817/466572374", 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.118:6819/466572374", 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.118:6823/466572374", 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.118:6821/466572374", 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.810277+0000", 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.686068+0000", 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.949768+0000", 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:06.773303+0000", 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.009610+0000", 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-21T23:39:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540701547738038271, 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-21T23:37:07.059224+0000", 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: 5, 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [ 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "snaps": [ 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "begin": 4, 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "length": 1 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "begin": 6, 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "length": 2 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-21T23:39:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-21T23:39:29.373 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:29.375 INFO:tasks.thrashosds.thrasher:j is {'epoch': 94, 'fsid': '66973394-6d88-4f14-bf11-7ccd7c9cf4a7', 'created': '2024-09-21T23:36:58.086363+0000', 'modified': '2024-09-21T23:39:29.126368+0000', 'last_up_change': '2024-09-21T23:37:09.181667+0000', 'last_in_change': '2024-09-21T23:36:59.825771+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:37:11.018666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9277217388153076, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:37:15.320283+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9277217388153076, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': '638e3e48-1763-4ec9-8c06-634e0b3b793b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 915397992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 915397992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 915397992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 915397992}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 915397992}]}, 'public_addr': '172.21.15.79:6801/915397992', 'cluster_addr': '172.21.15.79:6803/915397992', 'heartbeat_back_addr': '172.21.15.79:6807/915397992', 'heartbeat_front_addr': '172.21.15.79:6805/915397992', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02967526-d5c5-4f68-8e25-343311dbaec4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.695556640625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 3900686792}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 3900686792}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 3900686792}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3900686792}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 3900686792}]}, 'public_addr': '172.21.15.79:6809/3900686792', 'cluster_addr': '172.21.15.79:6811/3900686792', 'heartbeat_back_addr': '172.21.15.79:6815/3900686792', 'heartbeat_front_addr': '172.21.15.79:6813/3900686792', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4f770ac1-d062-4934-81d5-7f980b3830a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8707733154296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2564679849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 2564679849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 2564679849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 2564679849}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 2564679849}]}, 'public_addr': '172.21.15.79:6817/2564679849', 'cluster_addr': '172.21.15.79:6819/2564679849', 'heartbeat_back_addr': '172.21.15.79:6823/2564679849', 'heartbeat_front_addr': '172.21.15.79:6821/2564679849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '90cc3313-3845-48d0-839d-8fea4c52bfde', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 1410364236}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 1410364236}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 1410364236}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 1410364236}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 1410364236}]}, 'public_addr': '172.21.15.118:6809/1410364236', 'cluster_addr': '172.21.15.118:6811/1410364236', 'heartbeat_back_addr': '172.21.15.118:6815/1410364236', 'heartbeat_front_addr': '172.21.15.118:6813/1410364236', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '42b8e641-af88-4d41-bc5f-a14020f205ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 1572346429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 1572346429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 1572346429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 1572346429}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 1572346429}]}, 'public_addr': '172.21.15.118:6801/1572346429', 'cluster_addr': '172.21.15.118:6803/1572346429', 'heartbeat_back_addr': '172.21.15.118:6807/1572346429', 'heartbeat_front_addr': '172.21.15.118:6805/1572346429', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6817', 'nonce': 466572374}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6818', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 466572374}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6822', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 466572374}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 466572374}, {'type': 'v1', 'addr': '172.21.15.118:6821', 'nonce': 466572374}]}, 'public_addr': '172.21.15.118:6817/466572374', 'cluster_addr': '172.21.15.118:6819/466572374', 'heartbeat_back_addr': '172.21.15.118:6823/466572374', 'heartbeat_front_addr': '172.21.15.118:6821/466572374', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.810277+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.686068+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.949768+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:06.773303+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.009610+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:37:07.059224+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.0', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 4, 'length': 1}, {'begin': 6, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:39:29.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:29.376 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.376 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:39:29.429 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-21T23:39:29.429+0000 7fb5b553d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25272) UID: 0 2024-09-21T23:39:29.491 INFO:tasks.ceph.osd.4.smithi118.stderr:2024-09-21T23:39:29.490+0000 7f2cf66a7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25327) UID: 0 2024-09-21T23:39:29.537 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:39:29.601 INFO:tasks.ceph.osd.5.smithi118.stderr:2024-09-21T23:39:29.602+0000 7ff1cb039640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25328) UID: 0 2024-09-21T23:39:29.683 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:29.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T23:39:29.684 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.684 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.692 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-21T23:39:29.693+0000 7fbed8677640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25267) UID: 0 2024-09-21T23:39:29.695 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T23:39:29.695 DEBUG:teuthology.orchestra.run.smithi165:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-21T23:39:29.699 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:39:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:29.707 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:29.723 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-21T23:39:29.133388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147464939,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":12,"num_read":7065,"num_read_kb":180924,"num_write":1041,"num_write_kb":275345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1086,"ondisk_log_size":1086,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":481668,"kb_used_data":282024,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":565749372,"statfs":{"total":579820584960,"available":579327356928,"internally_reserved":0,"allocated":288792576,"data_stored":293335076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-70,"num_objects":-6,"num_object_clones":0,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":0,"num_read":-38,"num_read_kb":-30,"num_write":-6,"num_write_kb":-6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-19,"num_keys_recovered":-8,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026931"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":309,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.096868+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:24.096868+0000","last_peered":"2024-09-21T23:39:24.096868+0000","last_clean":"2024-09-21T23:39:24.096868+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:24.096868+0000","last_undegraded":"2024-09-21T23:39:24.096868+0000","last_fullsized":"2024-09-21T23:39:24.096868+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'121","reported_seq":1213,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.127479+0000","last_change":"2024-09-21T23:39:24.127479+0000","last_active":"2024-09-21T23:39:24.127479+0000","last_peered":"2024-09-21T23:39:24.127479+0000","last_clean":"2024-09-21T23:39:24.127479+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:24.127479+0000","last_undegraded":"2024-09-21T23:39:24.127479+0000","last_fullsized":"2024-09-21T23:39:24.127479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:16:44.738542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017152545000000002,"stat_sum":{"num_bytes":20971574,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1162,"num_read_kb":45717,"num_write":86,"num_write_kb":49221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.0","version":"89'152","reported_seq":777,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.119301+0000","last_change":"2024-09-21T23:39:24.119583+0000","last_active":"2024-09-21T23:39:24.119301+0000","last_peered":"2024-09-21T23:39:24.119301+0000","last_clean":"2024-09-21T23:39:24.119301+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:24.119301+0000","last_undegraded":"2024-09-21T23:39:24.119301+0000","last_fullsized":"2024-09-21T23:39:24.119301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:06:23.149381+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0093304849999999995,"stat_sum":{"num_bytes":20992000,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":511,"num_read_kb":30664,"num_write":137,"num_write_kb":41035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.1","version":"83'63","reported_seq":516,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.116759+0000","last_change":"2024-09-21T23:39:24.116759+0000","last_active":"2024-09-21T23:39:24.116759+0000","last_peered":"2024-09-21T23:39:24.116759+0000","last_clean":"2024-09-21T23:39:24.116759+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:24.116759+0000","last_undegraded":"2024-09-21T23:39:24.116759+0000","last_fullsized":"2024-09-21T23:39:24.116759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:34:36.087140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034138300000000001,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":53,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.2","version":"89'53","reported_seq":421,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.119781+0000","last_change":"2024-09-21T23:39:24.119781+0000","last_active":"2024-09-21T23:39:24.119781+0000","last_peered":"2024-09-21T23:39:24.119781+0000","last_clean":"2024-09-21T23:39:24.119781+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:24.119781+0000","last_undegraded":"2024-09-21T23:39:24.119781+0000","last_fullsized":"2024-09-21T23:39:24.119781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:43.881570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095507579999999995,"stat_sum":{"num_bytes":20971539,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":199,"num_read_kb":24705,"num_write":43,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.4","version":"83'6","reported_seq":214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.100297+0000","last_change":"2024-09-21T23:39:24.102684+0000","last_active":"2024-09-21T23:39:24.100297+0000","last_peered":"2024-09-21T23:39:24.100297+0000","last_clean":"2024-09-21T23:39:24.100297+0000","last_became_active":"2024-09-21T23:39:13.710444+0000","last_became_peered":"2024-09-21T23:39:13.710444+0000","last_unstale":"2024-09-21T23:39:24.100297+0000","last_undegraded":"2024-09-21T23:39:24.100297+0000","last_fullsized":"2024-09-21T23:39:24.100297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:54:09.802390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002620917,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":18,"num_write":5,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.5","version":"89'207","reported_seq":643,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.119021+0000","last_change":"2024-09-21T23:39:24.105982+0000","last_active":"2024-09-21T23:39:24.119021+0000","last_peered":"2024-09-21T23:39:24.119021+0000","last_clean":"2024-09-21T23:39:24.119021+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:24.119021+0000","last_undegraded":"2024-09-21T23:39:24.119021+0000","last_fullsized":"2024-09-21T23:39:24.119021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:27:34.377277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089919160000000008,"stat_sum":{"num_bytes":20971612,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":998,"num_read_kb":19603,"num_write":301,"num_write_kb":37083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.6","version":"89'105","reported_seq":392,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.110805+0000","last_change":"2024-09-21T23:39:24.111205+0000","last_active":"2024-09-21T23:39:24.110805+0000","last_peered":"2024-09-21T23:39:24.110805+0000","last_clean":"2024-09-21T23:39:24.110805+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:24.110805+0000","last_undegraded":"2024-09-21T23:39:24.110805+0000","last_fullsized":"2024-09-21T23:39:24.110805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:41.019812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010632667,"stat_sum":{"num_bytes":21020710,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":233,"num_read_kb":24681,"num_write":105,"num_write_kb":45143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]},{"pgid":"2.7","version":"89'262","reported_seq":1522,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T23:39:24.116743+0000","last_change":"2024-09-21T23:39:24.101612+0000","last_active":"2024-09-21T23:39:24.116743+0000","last_peered":"2024-09-21T23:39:24.116743+0000","last_clean":"2024-09-21T23:39:24.116743+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:24.116743+0000","last_undegraded":"2024-09-21T23:39:24.116743+0000","last_fullsized":"2024-09-21T23:39:24.116743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:45.030051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045861549999999997,"stat_sum":{"num_bytes":25169920,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":3441,"num_read_kb":32554,"num_write":152,"num_write_kb":47157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":146874571,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":12,"num_read":6949,"num_read_kb":180824,"num_write":882,"num_write_kb":272929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":281362432,"data_stored":289751446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607583,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93800,"kb_used_data":58444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94278040,"statfs":{"total":96636764160,"available":96540712960,"internally_reserved":0,"allocated":59846656,"data_stored":63372436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607584,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":23560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":24125440,"data_stored":23534683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607583,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100528,"kb_used_data":65164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94271312,"statfs":{"total":96636764160,"available":96533823488,"internally_reserved":0,"allocated":66727936,"data_stored":66117376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607583,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81436,"kb_used_data":50248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94290404,"statfs":{"total":96636764160,"available":96553373696,"internally_reserved":0,"allocated":51453952,"data_stored":54983809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607582,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89608,"kb_used_data":58420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282232,"statfs":{"total":96636764160,"available":96545005568,"internally_reserved":0,"allocated":59822080,"data_stored":59169978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607583,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57388,"kb_used_data":26188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314452,"statfs":{"total":96636764160,"available":96577998848,"internally_reserved":0,"allocated":26816512,"data_stored":26156794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25178112,"data_stored":25165916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58777600,"data_stored":58769465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50417664,"data_stored":54583350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65093632,"data_stored":65126530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23085056,"data_stored":23134208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":58810368,"data_stored":62971977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:29.724 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [5, 3, 1, 0] 2024-09-21T23:39:29.724 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.0', '5', '3', '1', '0'] 2024-09-21T23:39:29.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.0 5 3 1 0 2024-09-21T23:39:29.751 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T23:39:29.751 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T23:39:29.760 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T23:39:29.866 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.866 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:29.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:39:29.996 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:29.997 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:39:30.135 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.135 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.144 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-21T23:39:30.144 INFO:teuthology.orchestra.run.smithi079.stderr:set 1.0 pg_upmap_items mapping to [5->3,1->0] 2024-09-21T23:39:30.151 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:39:30.271 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.272 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.286 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:39:30.402 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.402 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.416 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:39:30.539 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.539 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.553 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:39:30.671 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.671 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:30.686 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:39:30.807 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-21T23:39:30.807 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:39:32.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:39:32.839 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-21T23:39:32.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:39:34.162 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-21T23:39:35.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T23:39:35.307 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-21T23:39:35.307 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:39:35.320 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-21T23:39:35.320 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:39:35.330 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-21T23:39:35.330 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:39:35.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:39:36.190 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:39:36.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T23:39:37.186 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T23:39:37.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:39:38.198 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:39:38.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T23:39:39.212 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T23:39:39.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:39:40.224 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:39:40.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T23:39:41.238 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T23:39:41.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:39:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:39:42.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T23:39:42.612 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-21T23:39:42.613+0000 7fade22fc640 0 --2- 172.21.15.79:0/277607470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faddc0a3810 0x7faddc11fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T23:39:43.266 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T23:39:43.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:39:44.280 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:39:44.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T23:39:45.282 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T23:39:45.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:39:46.286 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:39:46.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T23:39:47.302 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T23:39:47.330 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T23:39:47.330 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T23:39:47.330 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T23:39:47.330 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T23:39:47.331 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:39:47.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:39:47.629 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:47.629 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":109,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:39:47.296483+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:37:15.320283+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:39:47.647 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:39:47.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:39:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:47.930 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":109,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:39:47.296483+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:37:15.320283+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:39:47.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:39:47.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:39:47.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:39:47.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:39:47.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:39:47.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:39:48.122 INFO:teuthology.orchestra.run.smithi079.stdout:51539607589 2024-09-21T23:39:48.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:39:48.125 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:48.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:39:48.169 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:48.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:39:48.180 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:48.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:39:48.194 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:48.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:39:48.197 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:48.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:39:48.494 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:48.496 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:48.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607589 got 51539607588 for osd.4 2024-09-21T23:39:48.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607587 for osd.2 2024-09-21T23:39:48.582 INFO:teuthology.orchestra.run.smithi079.stdout:51539607586 2024-09-21T23:39:48.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607587 got 51539607586 for osd.1 2024-09-21T23:39:48.612 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:48.620 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:48.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607587 for osd.0 2024-09-21T23:39:48.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607587 for osd.3 2024-09-21T23:39:48.674 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:48.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607587 for osd.5 2024-09-21T23:39:49.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:39:49.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:39:49.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:39:49.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:39:49.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:39:49.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:39:49.905 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:49.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607588 for osd.2 2024-09-21T23:39:49.930 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:49.942 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:49.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607588 for osd.0 2024-09-21T23:39:49.959 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:49.979 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:49.988 INFO:teuthology.orchestra.run.smithi079.stdout:51539607589 2024-09-21T23:39:49.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607588 for osd.3 2024-09-21T23:39:50.000 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:50.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607589 got 51539607589 for osd.4 2024-09-21T23:39:50.010 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:50.032 INFO:teuthology.orchestra.run.smithi079.stdout:51539607587 2024-09-21T23:39:50.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607587 got 51539607587 for osd.1 2024-09-21T23:39:50.050 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:50.066 INFO:teuthology.orchestra.run.smithi079.stdout:51539607588 2024-09-21T23:39:50.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607588 got 51539607588 for osd.5 2024-09-21T23:39:50.082 DEBUG:teuthology.parallel:result is None 2024-09-21T23:39:50.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:39:50.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:50.394 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:50.411 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-21T23:39:48.989471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208292,"kb_used_data":8656,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022748,"statfs":{"total":579820584960,"available":579607293952,"internally_reserved":0,"allocated":8863744,"data_stored":4139414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.711702"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":347,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292325+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:46.292325+0000","last_peered":"2024-09-21T23:39:46.292325+0000","last_clean":"2024-09-21T23:39:46.292325+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:46.292325+0000","last_undegraded":"2024-09-21T23:39:46.292325+0000","last_fullsized":"2024-09-21T23:39:46.292325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":572,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.598713+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:46.598713+0000","last_peered":"2024-09-21T23:39:46.598713+0000","last_clean":"2024-09-21T23:39:46.598713+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:46.598713+0000","last_undegraded":"2024-09-21T23:39:46.598713+0000","last_fullsized":"2024-09-21T23:39:46.598713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":727,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292363+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:46.292363+0000","last_peered":"2024-09-21T23:39:46.292363+0000","last_clean":"2024-09-21T23:39:46.292363+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:46.292363+0000","last_undegraded":"2024-09-21T23:39:46.292363+0000","last_fullsized":"2024-09-21T23:39:46.292363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1662,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292422+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:46.292422+0000","last_peered":"2024-09-21T23:39:46.292422+0000","last_clean":"2024-09-21T23:39:46.292422+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:46.292422+0000","last_undegraded":"2024-09-21T23:39:46.292422+0000","last_fullsized":"2024-09-21T23:39:46.292422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1277952,"data_stored":491590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607589,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":491571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607587,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1867776,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:50.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:50.681 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:50.682 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:50.698 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-21T23:39:48.989471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208292,"kb_used_data":8656,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022748,"statfs":{"total":579820584960,"available":579607293952,"internally_reserved":0,"allocated":8863744,"data_stored":4139414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.711702"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":347,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292325+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:46.292325+0000","last_peered":"2024-09-21T23:39:46.292325+0000","last_clean":"2024-09-21T23:39:46.292325+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:46.292325+0000","last_undegraded":"2024-09-21T23:39:46.292325+0000","last_fullsized":"2024-09-21T23:39:46.292325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":572,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.598713+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:46.598713+0000","last_peered":"2024-09-21T23:39:46.598713+0000","last_clean":"2024-09-21T23:39:46.598713+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:46.598713+0000","last_undegraded":"2024-09-21T23:39:46.598713+0000","last_fullsized":"2024-09-21T23:39:46.598713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":727,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292363+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:46.292363+0000","last_peered":"2024-09-21T23:39:46.292363+0000","last_clean":"2024-09-21T23:39:46.292363+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:46.292363+0000","last_undegraded":"2024-09-21T23:39:46.292363+0000","last_fullsized":"2024-09-21T23:39:46.292363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1662,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-21T23:39:46.292422+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:46.292422+0000","last_peered":"2024-09-21T23:39:46.292422+0000","last_clean":"2024-09-21T23:39:46.292422+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:46.292422+0000","last_undegraded":"2024-09-21T23:39:46.292422+0000","last_fullsized":"2024-09-21T23:39:46.292422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1277952,"data_stored":491590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607589,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":491571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607587,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607588,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1867776,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:50.699 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:39:50.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T23:39:50.999 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T23:39:51.009 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:39:51.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:51.300 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:51.318 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-21T23:39:50.990599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208340,"kb_used_data":8704,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022700,"statfs":{"total":579820584960,"available":579607244800,"internally_reserved":0,"allocated":8912896,"data_stored":4157842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002261"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455560+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:48.455560+0000","last_peered":"2024-09-21T23:39:48.455560+0000","last_clean":"2024-09-21T23:39:48.455560+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:48.455560+0000","last_undegraded":"2024-09-21T23:39:48.455560+0000","last_fullsized":"2024-09-21T23:39:48.455560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":574,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.192704+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:48.192704+0000","last_peered":"2024-09-21T23:39:48.192704+0000","last_clean":"2024-09-21T23:39:48.192704+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:48.192704+0000","last_undegraded":"2024-09-21T23:39:48.192704+0000","last_fullsized":"2024-09-21T23:39:48.192704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":729,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455553+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:48.455553+0000","last_peered":"2024-09-21T23:39:48.455553+0000","last_clean":"2024-09-21T23:39:48.455553+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:48.455553+0000","last_undegraded":"2024-09-21T23:39:48.455553+0000","last_fullsized":"2024-09-21T23:39:48.455553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1664,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455583+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:48.455583+0000","last_peered":"2024-09-21T23:39:48.455583+0000","last_clean":"2024-09-21T23:39:48.455583+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:48.455583+0000","last_undegraded":"2024-09-21T23:39:48.455583+0000","last_fullsized":"2024-09-21T23:39:48.455583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607589,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:51.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:51.611 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:51.628 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-21T23:39:50.990599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208340,"kb_used_data":8704,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022700,"statfs":{"total":579820584960,"available":579607244800,"internally_reserved":0,"allocated":8912896,"data_stored":4157842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002261"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455560+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:48.455560+0000","last_peered":"2024-09-21T23:39:48.455560+0000","last_clean":"2024-09-21T23:39:48.455560+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:48.455560+0000","last_undegraded":"2024-09-21T23:39:48.455560+0000","last_fullsized":"2024-09-21T23:39:48.455560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":574,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.192704+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:48.192704+0000","last_peered":"2024-09-21T23:39:48.192704+0000","last_clean":"2024-09-21T23:39:48.192704+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:48.192704+0000","last_undegraded":"2024-09-21T23:39:48.192704+0000","last_fullsized":"2024-09-21T23:39:48.192704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":729,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455553+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:48.455553+0000","last_peered":"2024-09-21T23:39:48.455553+0000","last_clean":"2024-09-21T23:39:48.455553+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:48.455553+0000","last_undegraded":"2024-09-21T23:39:48.455553+0000","last_fullsized":"2024-09-21T23:39:48.455553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1664,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455583+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:48.455583+0000","last_peered":"2024-09-21T23:39:48.455583+0000","last_clean":"2024-09-21T23:39:48.455583+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:48.455583+0000","last_undegraded":"2024-09-21T23:39:48.455583+0000","last_fullsized":"2024-09-21T23:39:48.455583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607589,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:51.629 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:39:51.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:51.899 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:51.900 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:51.915 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-21T23:39:50.990599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208340,"kb_used_data":8704,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022700,"statfs":{"total":579820584960,"available":579607244800,"internally_reserved":0,"allocated":8912896,"data_stored":4157842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002261"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455560+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:48.455560+0000","last_peered":"2024-09-21T23:39:48.455560+0000","last_clean":"2024-09-21T23:39:48.455560+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:48.455560+0000","last_undegraded":"2024-09-21T23:39:48.455560+0000","last_fullsized":"2024-09-21T23:39:48.455560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":574,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.192704+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:48.192704+0000","last_peered":"2024-09-21T23:39:48.192704+0000","last_clean":"2024-09-21T23:39:48.192704+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:48.192704+0000","last_undegraded":"2024-09-21T23:39:48.192704+0000","last_fullsized":"2024-09-21T23:39:48.192704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":729,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455553+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:48.455553+0000","last_peered":"2024-09-21T23:39:48.455553+0000","last_clean":"2024-09-21T23:39:48.455553+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:48.455553+0000","last_undegraded":"2024-09-21T23:39:48.455553+0000","last_fullsized":"2024-09-21T23:39:48.455553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1664,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455583+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:48.455583+0000","last_peered":"2024-09-21T23:39:48.455583+0000","last_clean":"2024-09-21T23:39:48.455583+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:48.455583+0000","last_undegraded":"2024-09-21T23:39:48.455583+0000","last_fullsized":"2024-09-21T23:39:48.455583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607589,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:51.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:39:52.201 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:52.201 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":109,"fsid":"66973394-6d88-4f14-bf11-7ccd7c9cf4a7","created":"2024-09-21T23:36:58.086363+0000","modified":"2024-09-21T23:39:47.296483+0000","last_up_change":"2024-09-21T23:37:09.181667+0000","last_in_change":"2024-09-21T23:36:59.825771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:37:11.018666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:37:15.320283+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"638e3e48-1763-4ec9-8c06-634e0b3b793b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6801","nonce":915397992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6803","nonce":915397992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6807","nonce":915397992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":915397992},{"type":"v1","addr":"172.21.15.79:6805","nonce":915397992}]},"public_addr":"172.21.15.79:6801/915397992","cluster_addr":"172.21.15.79:6803/915397992","heartbeat_back_addr":"172.21.15.79:6807/915397992","heartbeat_front_addr":"172.21.15.79:6805/915397992","state":["exists","up"]},{"osd":1,"uuid":"02967526-d5c5-4f68-8e25-343311dbaec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6809","nonce":3900686792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6811","nonce":3900686792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6815","nonce":3900686792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3900686792},{"type":"v1","addr":"172.21.15.79:6813","nonce":3900686792}]},"public_addr":"172.21.15.79:6809/3900686792","cluster_addr":"172.21.15.79:6811/3900686792","heartbeat_back_addr":"172.21.15.79:6815/3900686792","heartbeat_front_addr":"172.21.15.79:6813/3900686792","state":["exists","up"]},{"osd":2,"uuid":"4f770ac1-d062-4934-81d5-7f980b3830a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6817","nonce":2564679849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6819","nonce":2564679849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6823","nonce":2564679849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2564679849},{"type":"v1","addr":"172.21.15.79:6821","nonce":2564679849}]},"public_addr":"172.21.15.79:6817/2564679849","cluster_addr":"172.21.15.79:6819/2564679849","heartbeat_back_addr":"172.21.15.79:6823/2564679849","heartbeat_front_addr":"172.21.15.79:6821/2564679849","state":["exists","up"]},{"osd":3,"uuid":"90cc3313-3845-48d0-839d-8fea4c52bfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6809","nonce":1410364236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6811","nonce":1410364236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6815","nonce":1410364236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":1410364236},{"type":"v1","addr":"172.21.15.118:6813","nonce":1410364236}]},"public_addr":"172.21.15.118:6809/1410364236","cluster_addr":"172.21.15.118:6811/1410364236","heartbeat_back_addr":"172.21.15.118:6815/1410364236","heartbeat_front_addr":"172.21.15.118:6813/1410364236","state":["exists","up"]},{"osd":4,"uuid":"42b8e641-af88-4d41-bc5f-a14020f205ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6801","nonce":1572346429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6803","nonce":1572346429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6807","nonce":1572346429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":1572346429},{"type":"v1","addr":"172.21.15.118:6805","nonce":1572346429}]},"public_addr":"172.21.15.118:6801/1572346429","cluster_addr":"172.21.15.118:6803/1572346429","heartbeat_back_addr":"172.21.15.118:6807/1572346429","heartbeat_front_addr":"172.21.15.118:6805/1572346429","state":["exists","up"]},{"osd":5,"uuid":"2f82efcc-7002-4d8c-a3ea-60c5a8ba42e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6817","nonce":466572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6819","nonce":466572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6823","nonce":466572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":466572374},{"type":"v1","addr":"172.21.15.118:6821","nonce":466572374}]},"public_addr":"172.21.15.118:6817/466572374","cluster_addr":"172.21.15.118:6819/466572374","heartbeat_back_addr":"172.21.15.118:6823/466572374","heartbeat_front_addr":"172.21.15.118:6821/466572374","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.810277+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.686068+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.949768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:06.773303+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.009610+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:37:07.059224+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[5,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:39:53.219 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T23:39:53.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:53.375 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:53.376 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:53.376 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:53.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T23:39:53.671 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T23:39:53.689 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T23:39:53.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:53.832 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:53.832 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:53.832 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:53.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T23:39:54.130 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T23:39:54.147 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T23:39:54.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:54.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:54.278 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:54.278 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:54.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T23:39:54.562 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T23:39:54.579 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T23:39:54.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:54.716 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:54.716 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:54.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T23:39:55.001 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T23:39:55.018 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T23:39:55.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:55.152 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:55.152 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:55.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T23:39:55.430 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T23:39:55.447 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T23:39:55.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:39:55.585 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-21T23:39:55.586 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:39:55.586 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-21T23:39:55.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T23:39:55.895 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T23:39:55.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:39:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:39:56.207 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:39:56.225 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-21T23:39:54.991556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1230,"ondisk_log_size":1230,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208340,"kb_used_data":8704,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022700,"statfs":{"total":579820584960,"available":579607244800,"internally_reserved":0,"allocated":8912896,"data_stored":4157842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.702025"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455560+0000","last_change":"2024-09-21T23:37:12.941792+0000","last_active":"2024-09-21T23:39:48.455560+0000","last_peered":"2024-09-21T23:39:48.455560+0000","last_clean":"2024-09-21T23:39:48.455560+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:48.455560+0000","last_undegraded":"2024-09-21T23:39:48.455560+0000","last_fullsized":"2024-09-21T23:39:48.455560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:11.198916+0000","last_clean_scrub_stamp":"2024-09-21T23:37:11.198916+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:03.767881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'157","reported_seq":1387,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770194+0000","last_change":"2024-09-21T23:39:27.849843+0000","last_active":"2024-09-21T23:39:47.770194+0000","last_peered":"2024-09-21T23:39:47.770194+0000","last_clean":"2024-09-21T23:39:47.770194+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:47.770194+0000","last_undegraded":"2024-09-21T23:39:47.770194+0000","last_fullsized":"2024-09-21T23:39:47.770194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:37:43.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":851,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770158+0000","last_change":"2024-09-21T23:39:27.848537+0000","last_active":"2024-09-21T23:39:47.770158+0000","last_peered":"2024-09-21T23:39:47.770158+0000","last_clean":"2024-09-21T23:39:47.770158+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:47.770158+0000","last_undegraded":"2024-09-21T23:39:47.770158+0000","last_fullsized":"2024-09-21T23:39:47.770158+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:21.604826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":574,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.192704+0000","last_change":"2024-09-21T23:39:27.853756+0000","last_active":"2024-09-21T23:39:48.192704+0000","last_peered":"2024-09-21T23:39:48.192704+0000","last_clean":"2024-09-21T23:39:48.192704+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:48.192704+0000","last_undegraded":"2024-09-21T23:39:48.192704+0000","last_fullsized":"2024-09-21T23:39:48.192704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:24.907593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"93'60","reported_seq":494,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.770195+0000","last_change":"2024-09-21T23:39:27.848939+0000","last_active":"2024-09-21T23:39:47.770195+0000","last_peered":"2024-09-21T23:39:47.770195+0000","last_clean":"2024-09-21T23:39:47.770195+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:47.770195+0000","last_undegraded":"2024-09-21T23:39:47.770195+0000","last_fullsized":"2024-09-21T23:39:47.770195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:56.827164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":265,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303509+0000","last_change":"2024-09-21T23:39:40.235041+0000","last_active":"2024-09-21T23:39:47.303509+0000","last_peered":"2024-09-21T23:39:47.303509+0000","last_clean":"2024-09-21T23:39:47.303509+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:47.303509+0000","last_undegraded":"2024-09-21T23:39:47.303509+0000","last_fullsized":"2024-09-21T23:39:47.303509+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:06.503994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":729,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455553+0000","last_change":"2024-09-21T23:39:27.843585+0000","last_active":"2024-09-21T23:39:48.455553+0000","last_peered":"2024-09-21T23:39:48.455553+0000","last_clean":"2024-09-21T23:39:48.455553+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:48.455553+0000","last_undegraded":"2024-09-21T23:39:48.455553+0000","last_fullsized":"2024-09-21T23:39:48.455553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:56.080337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":482,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:47.303569+0000","last_change":"2024-09-21T23:39:27.845680+0000","last_active":"2024-09-21T23:39:47.303569+0000","last_peered":"2024-09-21T23:39:47.303569+0000","last_clean":"2024-09-21T23:39:47.303569+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:47.303569+0000","last_undegraded":"2024-09-21T23:39:47.303569+0000","last_fullsized":"2024-09-21T23:39:47.303569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:45.284580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1664,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:48.455583+0000","last_change":"2024-09-21T23:39:27.842921+0000","last_active":"2024-09-21T23:39:48.455583+0000","last_peered":"2024-09-21T23:39:48.455583+0000","last_clean":"2024-09-21T23:39:48.455583+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:48.455583+0000","last_undegraded":"2024-09-21T23:39:48.455583+0000","last_fullsized":"2024-09-21T23:39:48.455583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:37:16.236570+0000","last_clean_scrub_stamp":"2024-09-21T23:37:16.236570+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:47.283432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1113,"ondisk_log_size":1113,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607589,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:39:56.228 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T23:37:11.198916+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T23:37:16.236570+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=37, tm_sec=16, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=39, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:39:56.229 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T23:40:16.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:40:16.537 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-21T23:40:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-21T23:40:16.554 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-21T23:40:14.996852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7494,"num_read_kb":181297,"num_write":1140,"num_write_kb":275371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1234,"ondisk_log_size":1234,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":208300,"kb_used_data":8704,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566022740,"statfs":{"total":579820584960,"available":579607285760,"internally_reserved":0,"allocated":8912896,"data_stored":4157842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":204266145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003114"},"pg_stats":[{"pgid":"1.0","version":"109'119","reported_seq":360,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:55.796897+0000","last_change":"2024-09-21T23:39:55.796989+0000","last_active":"2024-09-21T23:39:55.796897+0000","last_peered":"2024-09-21T23:39:55.796897+0000","last_clean":"2024-09-21T23:39:55.796897+0000","last_became_active":"2024-09-21T23:37:12.940034+0000","last_became_peered":"2024-09-21T23:37:12.940034+0000","last_unstale":"2024-09-21T23:39:55.796897+0000","last_undegraded":"2024-09-21T23:39:55.796897+0000","last_fullsized":"2024-09-21T23:39:55.796897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'119","last_scrub_stamp":"2024-09-21T23:39:55.796705+0000","last_deep_scrub":"109'119","last_deep_scrub_stamp":"2024-09-21T23:39:55.796705+0000","last_clean_scrub_stamp":"2024-09-21T23:39:55.796705+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:53.896483+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"109'158","reported_seq":1397,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:57.236313+0000","last_change":"2024-09-21T23:39:57.236381+0000","last_active":"2024-09-21T23:39:57.236313+0000","last_peered":"2024-09-21T23:39:57.236313+0000","last_clean":"2024-09-21T23:39:57.236313+0000","last_became_active":"2024-09-21T23:37:17.270558+0000","last_became_peered":"2024-09-21T23:37:17.270558+0000","last_unstale":"2024-09-21T23:39:57.236313+0000","last_undegraded":"2024-09-21T23:39:57.236313+0000","last_fullsized":"2024-09-21T23:39:57.236313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'158","last_scrub_stamp":"2024-09-21T23:39:57.236116+0000","last_deep_scrub":"109'158","last_deep_scrub_stamp":"2024-09-21T23:39:57.236116+0000","last_clean_scrub_stamp":"2024-09-21T23:39:57.236116+0000","objects_scrubbed":1,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:17:26.188552+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0019318149999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1362,"num_read_kb":45894,"num_write":108,"num_write_kb":49228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"93'167","reported_seq":861,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:58.249318+0000","last_change":"2024-09-21T23:39:58.249378+0000","last_active":"2024-09-21T23:39:58.249318+0000","last_peered":"2024-09-21T23:39:58.249318+0000","last_clean":"2024-09-21T23:39:58.249318+0000","last_became_active":"2024-09-21T23:37:55.958032+0000","last_became_peered":"2024-09-21T23:37:55.958032+0000","last_unstale":"2024-09-21T23:39:58.249318+0000","last_undegraded":"2024-09-21T23:39:58.249318+0000","last_fullsized":"2024-09-21T23:39:58.249318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'167","last_scrub_stamp":"2024-09-21T23:39:58.249098+0000","last_deep_scrub":"93'167","last_deep_scrub_stamp":"2024-09-21T23:39:58.249098+0000","last_clean_scrub_stamp":"2024-09-21T23:39:58.249098+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:09:50.759471+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00060344299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":518,"num_read_kb":30671,"num_write":145,"num_write_kb":41038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'67","reported_seq":583,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:55.187964+0000","last_change":"2024-09-21T23:39:55.187989+0000","last_active":"2024-09-21T23:39:55.187964+0000","last_peered":"2024-09-21T23:39:55.187964+0000","last_clean":"2024-09-21T23:39:55.187964+0000","last_became_active":"2024-09-21T23:37:17.265470+0000","last_became_peered":"2024-09-21T23:37:17.265470+0000","last_unstale":"2024-09-21T23:39:55.187964+0000","last_undegraded":"2024-09-21T23:39:55.187964+0000","last_fullsized":"2024-09-21T23:39:55.187964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'67","last_scrub_stamp":"2024-09-21T23:39:55.187881+0000","last_deep_scrub":"93'67","last_deep_scrub_stamp":"2024-09-21T23:39:55.187881+0000","last_clean_scrub_stamp":"2024-09-21T23:39:55.187881+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:08:28.852320+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001232737,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":371,"num_read_kb":2882,"num_write":56,"num_write_kb":12325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"109'61","reported_seq":504,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:56.261523+0000","last_change":"2024-09-21T23:39:56.261593+0000","last_active":"2024-09-21T23:39:56.261523+0000","last_peered":"2024-09-21T23:39:56.261523+0000","last_clean":"2024-09-21T23:39:56.261523+0000","last_became_active":"2024-09-21T23:37:17.270769+0000","last_became_peered":"2024-09-21T23:37:17.270769+0000","last_unstale":"2024-09-21T23:39:56.261523+0000","last_undegraded":"2024-09-21T23:39:56.261523+0000","last_fullsized":"2024-09-21T23:39:56.261523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'61","last_scrub_stamp":"2024-09-21T23:39:56.261328+0000","last_deep_scrub":"109'61","last_deep_scrub_stamp":"2024-09-21T23:39:56.261328+0000","last_clean_scrub_stamp":"2024-09-21T23:39:56.261328+0000","objects_scrubbed":1,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:29.723437+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010016789999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":211,"num_read_kb":24717,"num_write":46,"num_write_kb":36866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"93'9","reported_seq":274,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:55.263426+0000","last_change":"2024-09-21T23:39:55.263468+0000","last_active":"2024-09-21T23:39:55.263426+0000","last_peered":"2024-09-21T23:39:55.263426+0000","last_clean":"2024-09-21T23:39:55.263426+0000","last_became_active":"2024-09-21T23:39:40.234490+0000","last_became_peered":"2024-09-21T23:39:40.234490+0000","last_unstale":"2024-09-21T23:39:55.263426+0000","last_undegraded":"2024-09-21T23:39:55.263426+0000","last_fullsized":"2024-09-21T23:39:55.263426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'9","last_scrub_stamp":"2024-09-21T23:39:55.263337+0000","last_deep_scrub":"93'9","last_deep_scrub_stamp":"2024-09-21T23:39:55.263337+0000","last_clean_scrub_stamp":"2024-09-21T23:39:55.263337+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:43.794142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0012099299999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":24,"num_write":9,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"93'229","reported_seq":738,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:57.854061+0000","last_change":"2024-09-21T23:39:57.854126+0000","last_active":"2024-09-21T23:39:57.854061+0000","last_peered":"2024-09-21T23:39:57.854061+0000","last_clean":"2024-09-21T23:39:57.854061+0000","last_became_active":"2024-09-21T23:37:18.031126+0000","last_became_peered":"2024-09-21T23:37:18.031126+0000","last_unstale":"2024-09-21T23:39:57.854061+0000","last_undegraded":"2024-09-21T23:39:57.854061+0000","last_fullsized":"2024-09-21T23:39:57.854061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'229","last_scrub_stamp":"2024-09-21T23:39:57.853858+0000","last_deep_scrub":"93'229","last_deep_scrub_stamp":"2024-09-21T23:39:57.853858+0000","last_clean_scrub_stamp":"2024-09-21T23:39:57.853858+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:18.409403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1037,"num_read_kb":19629,"num_write":319,"num_write_kb":37088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'130","reported_seq":491,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:54.285270+0000","last_change":"2024-09-21T23:39:54.285300+0000","last_active":"2024-09-21T23:39:54.285270+0000","last_peered":"2024-09-21T23:39:54.285270+0000","last_clean":"2024-09-21T23:39:54.285270+0000","last_became_active":"2024-09-21T23:37:17.260586+0000","last_became_peered":"2024-09-21T23:37:17.260586+0000","last_unstale":"2024-09-21T23:39:54.285270+0000","last_undegraded":"2024-09-21T23:39:54.285270+0000","last_fullsized":"2024-09-21T23:39:54.285270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'130","last_scrub_stamp":"2024-09-21T23:39:54.285173+0000","last_deep_scrub":"93'130","last_deep_scrub_stamp":"2024-09-21T23:39:54.285173+0000","last_clean_scrub_stamp":"2024-09-21T23:39:54.285173+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:45:54.897370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012286269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":275,"num_read_kb":24715,"num_write":126,"num_write_kb":45149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"93'294","reported_seq":1673,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:39:56.831273+0000","last_change":"2024-09-21T23:39:56.831339+0000","last_active":"2024-09-21T23:39:56.831273+0000","last_peered":"2024-09-21T23:39:56.831273+0000","last_clean":"2024-09-21T23:39:56.831273+0000","last_became_active":"2024-09-21T23:37:17.265440+0000","last_became_peered":"2024-09-21T23:37:17.265440+0000","last_unstale":"2024-09-21T23:39:56.831273+0000","last_undegraded":"2024-09-21T23:39:56.831273+0000","last_fullsized":"2024-09-21T23:39:56.831273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'294","last_scrub_stamp":"2024-09-21T23:39:56.831044+0000","last_deep_scrub":"93'294","last_deep_scrub_stamp":"2024-09-21T23:39:56.831044+0000","last_clean_scrub_stamp":"2024-09-21T23:39:56.831044+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:04:42.807103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056795399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3564,"num_read_kb":32665,"num_write":172,"num_write_kb":47162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7378,"num_read_kb":181197,"num_write":981,"num_write_kb":272955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1115,"ondisk_log_size":1115,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36604,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335236,"statfs":{"total":96636764160,"available":96599281664,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607594,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607593,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":496178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607592,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1880064,"data_stored":1086546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:40:16.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T23:40:16.876 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T23:40:16.876 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T23:40:16.876 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T23:40:16.876 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:16.946 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T23:40:16.946 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T23:40:16.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.003 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T23:40:17.003 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T23:40:17.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.076 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T23:40:17.076 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T23:40:17.076 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.170 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T23:40:17.171 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T23:40:17.171 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.252 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T23:40:17.252 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T23:40:17.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.346 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T23:40:17.346 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T23:40:17.346 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T23:40:17.346 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.377 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T23:40:17.377 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T23:40:17.377 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T23:40:17.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.400 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T23:40:17.400 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T23:40:17.400 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.465 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T23:40:17.466 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T23:40:17.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:40:17.485 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T23:40:17.485 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T23:40:17.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-21T23:40:17.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:40:17.526 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T23:40:17.854 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:40:17.854 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T23:40:18.071 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:40:18.072 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T23:40:18.231 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:40:18.231 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T23:40:18.474 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:40:18.474 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T23:40:18.672 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:40:18.673 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T23:40:18.855 INFO:tasks.ceph:Archiving mon data... 2024-09-21T23:40:18.857 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/data/mon.a.tgz 2024-09-21T23:40:18.857 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-21T23:40:18.863 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.iDa5V0bSy9 2024-09-21T23:40:18.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.iDa5V0bSy9 2024-09-21T23:40:18.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.iDa5V0bSy9 2024-09-21T23:40:19.040 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.iDa5V0bSy9 is 255KB 2024-09-21T23:40:19.098 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.iDa5V0bSy9 2024-09-21T23:40:19.103 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/data/mon.c.tgz 2024-09-21T23:40:19.103 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-21T23:40:19.152 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.Wq0mHjEGed 2024-09-21T23:40:19.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Wq0mHjEGed 2024-09-21T23:40:19.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.Wq0mHjEGed 2024-09-21T23:40:19.348 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.Wq0mHjEGed is 356KB 2024-09-21T23:40:19.406 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.Wq0mHjEGed 2024-09-21T23:40:19.411 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/data/mon.b.tgz 2024-09-21T23:40:19.411 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-21T23:40:19.417 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.6ywQiWHAhT 2024-09-21T23:40:19.417 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6ywQiWHAhT 2024-09-21T23:40:19.549 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.6ywQiWHAhT 2024-09-21T23:40:19.614 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.6ywQiWHAhT is 356KB 2024-09-21T23:40:19.676 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.6ywQiWHAhT 2024-09-21T23:40:19.681 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T23:40:19.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:40:19.684 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:40:19.724 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:40:19.744 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:40:19.783 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:40:19.789 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:40:19.789 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T23:40:19.790 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi079/crash 2024-09-21T23:40:19.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:40:19.804 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi118/crash 2024-09-21T23:40:19.805 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:40:19.839 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi165/crash 2024-09-21T23:40:19.840 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:40:19.852 INFO:tasks.ceph:Compressing logs... 2024-09-21T23:40:19.852 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:40:19.855 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:40:19.877 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36674.log 2024-09-21T23:40:19.878 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36674.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.36674.log.gz --verbose 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr: -- /var/log/ceph/ceph-client.admin.32440.log 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31272.log 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34747.log 2024-09-21T23:40:19.879 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32440.log.gz 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31272.log.gz 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25360.log 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34747.log.gz 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34942.log 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25360.log: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.25360.log.gz 2024-09-21T23:40:19.881 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30088.log 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34942.log.gz 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31550.log 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30088.log.gz 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37175.log 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32414.log 2024-09-21T23:40:19.883 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31550.log.gz 2024-09-21T23:40:19.884 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37175.log.gz 2024-09-21T23:40:19.884 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37123.log 2024-09-21T23:40:19.890 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:40:19.893 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32414.log.gz 2024-09-21T23:40:19.893 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38033.log 2024-09-21T23:40:19.893 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32602.log 2024-09-21T23:40:19.893 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37123.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37305.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38033.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32602.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30903.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34064.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37305.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30903.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34064.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.34064.log.gz --verbose 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr: -- /var/log/ceph/ceph-client.admin.30379.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36294.log 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-09-21T23:40:19.894 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35599.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30379.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36294.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35599.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35599.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32988.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35975.log 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32988.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35132.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz 2024-09-21T23:40:19.895 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.36581.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34475.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35975.log.gz 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36774.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32929.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34475.log.gz 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36774.log.gz 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35485.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.32929.log: -- /var/log/ceph/ceph-client.admin.36080.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32929.log.gz 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35485.log.gzgzip 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34450.log 2024-09-21T23:40:19.896 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36626.log 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36080.log.gz 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31330.log 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34450.log.gz 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32507.log 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36626.log.gz 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-09-21T23:40:19.897 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31330.log.gz 2024-09-21T23:40:19.918 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25227.log 2024-09-21T23:40:19.918 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25251.log 2024-09-21T23:40:19.918 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25179.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25227.log.gz 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25251.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25251.log.gz 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25179.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 0.0% 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25179.log.gz 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25299.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25299.log.gz 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:40:19.919 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25275.log 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25203.log 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25275.log.gz 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.25203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25203.log.gz 2024-09-21T23:40:19.920 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33766.log 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32507.log.gz 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33766.log.gz 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30996.log 2024-09-21T23:40:19.940 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33143.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30996.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33143.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30783.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35157.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31647.log 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30783.log.gz 2024-09-21T23:40:19.941 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35157.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31591.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31647.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30731.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31591.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36321.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30731.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34642.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36321.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.36321.log.gz /var/log/ceph/ceph-client.admin.34996.log 2024-09-21T23:40:19.942 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38189.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34642.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37149.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38189.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38189.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37149.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34215.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.35226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34314.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34215.log.gz 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33382.log 2024-09-21T23:40:19.943 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34314.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34819.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33382.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34819.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34038.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34038.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.37594.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30113.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38163.log 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30113.log.gz 2024-09-21T23:40:19.944 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36446.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32925.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38163.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36798.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36446.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36400.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32925.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33273.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36798.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30355.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36400.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34533.log 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33273.log.gz 2024-09-21T23:40:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30355.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34533.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34533.log.gz -5 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30063.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35361.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38059.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30063.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31144.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35361.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37617.log 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-09-21T23:40:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31144.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37479.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37617.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35294.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37479.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33014.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30555.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35294.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37981.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33014.log.gz 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31933.log 2024-09-21T23:40:19.947 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30555.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37981.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38449.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31933.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32028.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38449.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36257.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35773.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32028.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37476.log 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36257.log.gz 2024-09-21T23:40:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31756.log 2024-09-21T23:40:19.949 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35773.log.gz 2024-09-21T23:40:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-09-21T23:40:19.949 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37476.log.gz 2024-09-21T23:40:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37227.log 2024-09-21T23:40:19.949 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31756.log.gz 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37253.log 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30559.log 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37227.log.gz 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37253.log.gz 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30523.log 2024-09-21T23:40:19.950 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35534.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30559.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35802.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30523.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35534.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25276.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35802.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 0.0% 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31865.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25276.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32768.log 2024-09-21T23:40:19.951 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36232.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31865.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.32768.log: -- /var/log/ceph/ceph-client.admin.38007.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32768.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31455.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36232.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38007.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38007.log.gz -5 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31864.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31455.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37833.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.31864.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38085.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31864.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31705.log 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37833.log.gz 2024-09-21T23:40:19.952 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38085.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31997.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31705.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33337.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31997.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35095.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30679.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33337.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35095.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35095.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30679.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.31643.log -- replaced with /var/log/ceph/ceph-client.admin.30679.log.gz 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-21T23:40:19.953 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33473.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31643.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33865.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33473.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35302.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33865.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37929.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35302.log.gz 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-09-21T23:40:19.954 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37929.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37829.log: -- /var/log/ceph/ceph-client.admin.33436.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31095.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34815.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33436.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31045.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31095.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34600.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34815.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.31045.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30757.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31045.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36133.log 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34600.log.gz 2024-09-21T23:40:19.955 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30757.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30757.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.31408.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30185.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36133.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35674.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31408.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30185.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30216.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35674.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35460.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38397.log 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30216.log.gz 2024-09-21T23:40:19.956 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34252.log 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35460.log.gz 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35021.log 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38397.log.gz 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34252.log.gz 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32565.log 2024-09-21T23:40:19.957 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35021.log.gz 2024-09-21T23:40:19.964 INFO:teuthology.orchestra.run.smithi118.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T23:40:19.992 INFO:teuthology.orchestra.run.smithi118.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T23:40:20.002 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38473.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-09-21T23:40:20.002 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33544.log 2024-09-21T23:40:20.002 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-21T23:40:20.002 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32565.log.gz 2024-09-21T23:40:20.002 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33544.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35897.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33041.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30705.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33986.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33041.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32112.log 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30705.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:40:20.003 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31508.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33986.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32112.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32664.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31508.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37279.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33824.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32664.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30133.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 28.6% -- replaced with /var/log/ceph/ceph-client.admin.37279.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30527.log 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33824.log.gz 2024-09-21T23:40:20.004 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30585.log 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30133.log.gz 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33906.log 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30527.log.gz 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37040.log 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30585.log.gz 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38319.log 2024-09-21T23:40:20.005 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33906.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32076.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37040.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34875.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38319.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32076.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31239.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34875.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33960.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31819.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31239.log.gz 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33960.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21514.log 2024-09-21T23:40:20.006 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33960.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31819.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.tmp-client.admin.21514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35872.log 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21514.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31967.log 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31794.log 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35872.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31967.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31184.log 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31794.log.gz 2024-09-21T23:40:20.007 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30231.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30552.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31184.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32859.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30231.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30552.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34638.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32859.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30865.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33604.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34638.log: 0.0%/var/log/ceph/ceph-client.admin.30865.log: -- replaced with /var/log/ceph/ceph-client.admin.34638.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30228.log 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30865.log.gz 2024-09-21T23:40:20.008 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33604.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32190.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30228.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36498.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32190.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32326.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36898.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36498.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32326.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32326.log.gz 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36898.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.36898.log.gz -- 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.31617.log 2024-09-21T23:40:20.009 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30230.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.31617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29405.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31617.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30230.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32826.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30159.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29405.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32826.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36857.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37477.log 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30159.log.gz 2024-09-21T23:40:20.010 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36857.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36857.log.gz -5 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30835.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37477.log.gz 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31356.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30904.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30835.log.gz 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31356.log.gz 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30904.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.30904.log.gz --verbose 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: -- /var/log/ceph/ceph-client.admin.34702.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35971.log 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-09-21T23:40:20.011 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36701.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34575.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35971.log.gz 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30225.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36701.log.gz 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34777.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34575.log.gz 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30373.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30225.log.gz/var/log/ceph/ceph-client.admin.34777.log: 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34777.log.gz 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30406.log 2024-09-21T23:40:20.012 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30373.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25159.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32701.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30406.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37781.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25159.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32138.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32701.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37781.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33724.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32138.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33503.log 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-21T23:40:20.013 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32289.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33724.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38423.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33503.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38111.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32289.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38137.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38423.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33102.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38111.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37955.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38137.log.gz 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33699.log 2024-09-21T23:40:20.014 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33102.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32023.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37955.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34393.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33699.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32388.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32023.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34351.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34393.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37357.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32388.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34351.log: /var/log/ceph/ceph-client.admin.37357.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38293.log 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34351.log.gz 2024-09-21T23:40:20.015 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37357.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30222.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30407.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38293.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33180.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30222.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32164.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30407.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32454.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33180.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38241.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32164.log.gz 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37435.log 2024-09-21T23:40:20.016 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32454.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32730.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38241.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37201.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37435.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30809.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32730.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34012.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33242.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30809.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34012.log.gz 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38215.log 2024-09-21T23:40:20.017 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33242.log.gz 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38215.log.gz 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35565.log 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2024-09-21T23:40:20.018 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35565.log.gz 2024-09-21T23:40:20.020 INFO:teuthology.orchestra.run.smithi079.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T23:40:20.057 INFO:teuthology.orchestra.run.smithi118.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T23:40:20.069 INFO:teuthology.orchestra.run.smithi079.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T23:40:20.082 INFO:teuthology.orchestra.run.smithi079.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24820.log 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22947.log 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24820.log.gz 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21841.log 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21955.log 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22947.log: /var/log/ceph/ceph-client.admin.21841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22947.log.gz 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21841.log.gz 2024-09-21T23:40:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21859.log 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21955.log: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.21955.log.gz 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25064.log 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23665.log 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21859.log: /var/log/ceph/ceph-client.admin.25064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25091.log 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25064.log.gz 24.1% 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.21859.log.gz 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22680.log 19.4% 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23665.log.gz 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.25091.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23337.log 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25091.log.gz 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22680.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22680.log.gz 2024-09-21T23:40:20.094 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21606.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23337.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.23337.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22775.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21606.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23066.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23055.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22775.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23066.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23066.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24145.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21850.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23055.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24145.log.gz 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22876.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23278.log 2024-09-21T23:40:20.095 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21850.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24031.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22876.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23278.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22066.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22223.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24031.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22938.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.22066.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24616.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.22223.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24309.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22938.log.gz 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22196.log 2024-09-21T23:40:20.096 INFO:teuthology.orchestra.run.smithi165.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24616.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24309.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24309.log.gz -5 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr: --verbose -- /var/log/ceph/ceph-client.admin.24066.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22812.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22196.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25174.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24066.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22812.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24410.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22092.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25174.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23154.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24410.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22092.log.gz 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22149.log 2024-09-21T23:40:20.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22911.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23154.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25031.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22149.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23057.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22911.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23518.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.25031.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23243.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23057.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25153.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23518.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22249.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23243.log.gz 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22496.log 2024-09-21T23:40:20.098 INFO:teuthology.orchestra.run.smithi165.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.25153.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22249.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.22249.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22327.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23101.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22496.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24282.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22327.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23392.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23101.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22120.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24282.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22550.log 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.23392.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24198.log 0.0% 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.22120.log.gz 2024-09-21T23:40:20.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23959.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22550.log.gz 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23701.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24198.log.gz 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23959.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.23959.log.gz /var/log/ceph/ceph-client.admin.25257.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22255.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23701.log.gz 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24172.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25257.log.gz 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22644.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22255.log.gz 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24015.log 2024-09-21T23:40:20.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24172.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.24172.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24256.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22644.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23794.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24015.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23241.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24256.log: 45.0% -- replaced with /var/log/ceph/ceph-client.admin.24256.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24163.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23794.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23241.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.23181.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23241.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22481.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24163.log.gz 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23553.log 2024-09-21T23:40:20.101 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23181.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22305.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22481.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22014.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23553.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24247.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22305.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22042.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.22014.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23128.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24247.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.22042.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23663.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22042.log.gz 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24408.log 2024-09-21T23:40:20.102 INFO:teuthology.orchestra.run.smithi165.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23128.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24124.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23663.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24220.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24408.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24913.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24124.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24220.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23445.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23350.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24913.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21894.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23445.log.gz 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25112.log 2024-09-21T23:40:20.103 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23350.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24359.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21894.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25112.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21946.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24844.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24359.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23418.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21946.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24079.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24844.log.gz 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23418.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.23418.log.gz --verbose 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-client.admin.23119.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23325.log 2024-09-21T23:40:20.104 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24079.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24346.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23119.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23638.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23325.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23225.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24346.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24855.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23638.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23225.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23225.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22432.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23366.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.24855.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23544.log 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22432.log.gz 2024-09-21T23:40:20.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23366.log.gz 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24363.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24683.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23544.log.gz 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23454.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24363.log.gz 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21726.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24683.log.gz 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24040.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23454.log.gz 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22055.log 2024-09-21T23:40:20.106 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21726.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24320.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.admin.24040.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21930.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22055.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23603.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.admin.24320.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23881.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21930.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25007.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23603.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25291.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23881.log.gz 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25076.log 2024-09-21T23:40:20.107 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25007.log.gz 2024-09-21T23:40:20.108 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25291.log.gz 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23117.log 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22199.log 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25076.log.gz 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23117.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23117.log.gz /var/log/ceph/ceph-client.admin.24582.log 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23017.log 2024-09-21T23:40:20.109 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22199.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24245.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 3.6% -- replaced with /var/log/ceph/ceph-client.admin.24582.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25133.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23017.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25004.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24245.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24295.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.25133.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22603.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25004.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23767.log 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24295.log.gz 2024-09-21T23:40:20.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22212.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22603.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23471.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23767.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22212.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23580.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22777.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23471.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24937.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23580.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22777.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22777.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23298.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22531.log 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24937.log.gz 2024-09-21T23:40:20.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23298.log: gzip 26.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.23298.log.gz --verbose 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-client.admin.21626.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24207.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22531.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22352.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21626.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24161.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24207.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22352.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.23735.log -- replaced with /var/log/ceph/ceph-client.admin.22352.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22742.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24161.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24846.log 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23735.log.gz 2024-09-21T23:40:20.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22379.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22742.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24716.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24846.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24303.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.22379.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21933.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24716.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23961.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24303.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23166.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21933.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24750.log 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23961.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23166.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.23166.log.gz 2024-09-21T23:40:20.113 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22839.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22641.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24750.log.gz 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22839.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.22839.log.gz --verbose 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-client.admin.23941.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22057.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22641.log.gz 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23941.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.23941.log.gz 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: --verbose -- /var/log/ceph/ceph-client.admin.24549.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24210.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.22057.log.gz 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23654.log 2024-09-21T23:40:20.114 INFO:teuthology.orchestra.run.smithi165.stderr: 5.5% -- replaced with /var/log/ceph/ceph-client.admin.24549.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24075.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24210.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25195.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23654.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23163.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24075.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21776.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25195.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22689.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23163.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21985.log 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21776.log.gz 2024-09-21T23:40:20.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25022.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22689.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22412.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21985.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24482.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.25022.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24787.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22412.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22368.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24482.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22098.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24787.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23030.log 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22368.log.gz 2024-09-21T23:40:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24089.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22098.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22973.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23030.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22355.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24089.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21885.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22973.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21696.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22355.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21885.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23827.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23334.log 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr: 25.7% -- replaced with /var/log/ceph/ceph-client.admin.21696.log.gz 2024-09-21T23:40:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24881.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23827.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23334.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23899.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23092.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24881.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21838.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23899.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24291.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23092.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22577.log 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21838.log.gz 2024-09-21T23:40:20.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25020.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24291.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23509.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.22577.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23383.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25020.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23944.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23509.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21586.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23383.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22494.log 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23944.log.gz 2024-09-21T23:40:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22982.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21586.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23228.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22494.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21670.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22982.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22885.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23228.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21569.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21670.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22214.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22885.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24790.log 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21569.log.gz 2024-09-21T23:40:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23970.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22214.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24940.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24790.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22039.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 44.5% -- replaced with /var/log/ceph/ceph-client.admin.23970.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24419.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24940.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23008.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22039.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24029.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.24419.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22786.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23008.log.gz 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24098.log 2024-09-21T23:40:20.121 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24029.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22786.log: gzip -5 --verbose -- 24.7% /var/log/ceph/ceph-client.admin.23353.log -- replaced with /var/log/ceph/ceph-client.admin.22786.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25079.log 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.24098.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22566.log 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23353.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22277.log 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25079.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24886.log 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22566.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24011.log 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22277.log.gz 2024-09-21T23:40:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24515.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24886.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21650.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24011.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23601.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 4.5% -- replaced with /var/log/ceph/ceph-client.admin.24515.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23190.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21650.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23601.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.24355.log -- replaced with /var/log/ceph/ceph-client.admin.23601.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21752.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.23190.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25274.log 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24355.log.gz 2024-09-21T23:40:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24023.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 3.6% -- replaced with /var/log/ceph/ceph-client.admin.21752.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23252.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25274.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23020.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24023.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22715.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 23.5% -- replaced with /var/log/ceph/ceph-client.admin.23252.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24799.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23020.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22874.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22715.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24649.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24799.log.gz 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24383.log 2024-09-21T23:40:20.124 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22874.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24649.log: 5.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.24649.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.21785.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22920.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24383.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23706.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr: 27.5% -- replaced with /var/log/ceph/ceph-client.admin.21785.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23216.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22920.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22505.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr: 44.7% -- replaced with /var/log/ceph/ceph-client.admin.23706.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23216.log.gz 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23571.log 2024-09-21T23:40:20.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24957.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.22505.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24148.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23571.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25236.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.24957.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22453.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24148.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22849.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 51.4% -- replaced with /var/log/ceph/ceph-client.admin.25236.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21848.log 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22453.log.gz 2024-09-21T23:40:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22821.log 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22849.log.gz 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22478.log 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21848.log.gz 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21743.log 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22821.log.gz 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23287.log 2024-09-21T23:40:20.127 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22478.log.gz 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24955.log 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21743.log.gz 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22616.log 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23287.log.gz 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24955.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.24955.log.gz -- 2024-09-21T23:40:20.128 INFO:teuthology.orchestra.run.smithi165.stderr: /var/log/ceph/ceph-client.admin.22568.log 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22653.log 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr: 41.2% -- replaced with /var/log/ceph/ceph-client.admin.22616.log.gz 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21836.log 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22568.log.gz 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22370.log 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22653.log.gz 2024-09-21T23:40:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22678.log 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21836.log.gz 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23612.log 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.22370.log.gz 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23830.log 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22678.log.gz 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23179.log 2024-09-21T23:40:20.130 INFO:teuthology.orchestra.run.smithi165.stderr: 47.0% -- replaced with /var/log/ceph/ceph-client.admin.23612.log.gz 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23492.log 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23830.log.gz 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24992.log 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23179.log.gz 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23492.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.23492.log.gz -- 2024-09-21T23:40:20.131 INFO:teuthology.orchestra.run.smithi165.stderr: /var/log/ceph/ceph-client.admin.23427.log 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21903.log 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24992.log.gz 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23649.log 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23427.log.gz 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24087.log 2024-09-21T23:40:20.132 INFO:teuthology.orchestra.run.smithi165.stderr: 26.9% -- replaced with /var/log/ceph/ceph-client.admin.21903.log.gz 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24966.log 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23649.log.gz 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23996.log 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24087.log.gz 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23674.log 2024-09-21T23:40:20.133 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.24966.log.gz 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21687.log 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23996.log.gz 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23674.log: 62.1% -- replaced with /var/log/ceph/ceph-client.admin.23674.log.gz 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24446.log 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22171.log 2024-09-21T23:40:20.134 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21687.log.gz 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24446.log.gz 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23289.log 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22405.log 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22171.log.gz 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22724.log 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23289.log.gz 2024-09-21T23:40:20.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22553.log 0.0% 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.22405.log.gz 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23104.log 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22724.log.gz 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22553.log.gz 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22751.log 2024-09-21T23:40:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.23104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25216.log 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23104.log.gz 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24311.log 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22751.log.gz 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.25216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25216.log.gz 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21980.log 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.24311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22971.log 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24311.log.gz 2024-09-21T23:40:20.137 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.21980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21980.log.gz 2024-09-21T23:40:20.138 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22971.log.gz 2024-09-21T23:40:20.138 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T23:40:20.138 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m0.188s 2024-09-21T23:40:20.138 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m0.206s 2024-09-21T23:40:20.138 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.072s 2024-09-21T23:40:20.140 INFO:teuthology.orchestra.run.smithi079.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T23:40:20.140 INFO:teuthology.orchestra.run.smithi079.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T23:40:20.140 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-21T23:40:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m0.270s 2024-09-21T23:40:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m0.983s 2024-09-21T23:40:20.141 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.113s 2024-09-21T23:40:20.240 INFO:teuthology.orchestra.run.smithi118.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T23:40:20.300 INFO:teuthology.orchestra.run.smithi118.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T23:40:20.302 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-09-21T23:40:20.302 INFO:teuthology.orchestra.run.smithi118.stderr:real 0m0.415s 2024-09-21T23:40:20.302 INFO:teuthology.orchestra.run.smithi118.stderr:user 0m0.948s 2024-09-21T23:40:20.302 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m0.065s 2024-09-21T23:40:20.303 INFO:tasks.ceph:Archiving logs... 2024-09-21T23:40:20.303 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi079/log 2024-09-21T23:40:20.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:40:20.593 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi118/log 2024-09-21T23:40:20.593 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:40:20.729 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi165/log 2024-09-21T23:40:20.730 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:40:20.928 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T23:40:20.947 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T23:40:20.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:40:20.950 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:40:20.953 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:40:21.037 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T23:40:21.046 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T23:40:21.046 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:40:21.049 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:40:21.051 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:40:21.059 INFO:teuthology.orchestra.run.smithi079.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:40:21.059 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================== 2024-09-21T23:40:21.059 INFO:teuthology.orchestra.run.smithi079.stdout:*hv01.front.sepi 67.205.162.81 3 u 9 64 77 0.072 +0.974 2.779 2024-09-21T23:40:21.059 INFO:teuthology.orchestra.run.smithi079.stdout:+hv02.front.sepi 96.245.170.99 3 u 8 64 77 0.092 +1.573 1.600 2024-09-21T23:40:21.059 INFO:teuthology.orchestra.run.smithi079.stdout: hv03.front.sepi 74.6.168.72 3 u 11 64 77 0.084 -0.478 2.740 2024-09-21T23:40:21.060 INFO:teuthology.orchestra.run.smithi079.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:40:21.063 INFO:teuthology.orchestra.run.smithi165.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:40:21.063 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================== 2024-09-21T23:40:21.063 INFO:teuthology.orchestra.run.smithi165.stdout:+hv01.front.sepi 67.205.162.81 3 u 11 64 77 0.116 -0.724 2.373 2024-09-21T23:40:21.063 INFO:teuthology.orchestra.run.smithi165.stdout:*hv02.front.sepi 96.245.170.99 3 u 11 64 77 0.149 +2.209 2.492 2024-09-21T23:40:21.063 INFO:teuthology.orchestra.run.smithi165.stdout:+hv03.front.sepi 74.6.168.72 3 u 7 64 77 0.070 -2.343 2.266 2024-09-21T23:40:21.064 INFO:teuthology.orchestra.run.smithi165.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:40:21.064 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:40:21.064 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-09-21T23:40:21.064 INFO:teuthology.orchestra.run.smithi118.stdout:+hv01.front.sepi 67.205.162.81 3 u 13 64 77 0.108 -1.833 2.633 2024-09-21T23:40:21.064 INFO:teuthology.orchestra.run.smithi118.stdout:*hv02.front.sepi 96.245.170.99 3 u 13 64 77 0.117 +1.096 2.570 2024-09-21T23:40:21.065 INFO:teuthology.orchestra.run.smithi118.stdout:+hv03.front.sepi 74.6.168.72 3 u 8 64 77 0.125 -3.070 2.616 2024-09-21T23:40:21.065 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:40:21.065 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T23:40:21.074 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T23:40:21.074 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T23:40:21.107 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T23:40:21.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T23:40:21.173 INFO:teuthology.task.internal:Duration was 800.288683 seconds 2024-09-21T23:40:21.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T23:40:21.206 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T23:40:21.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:40:21.209 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:40:21.211 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:40:21.249 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T23:40:21.249 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-09-21T23:40:21.250 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:40:21.301 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-09-21T23:40:21.301 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:40:21.311 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-09-21T23:40:21.312 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:40:21.321 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T23:40:21.321 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:40:21.345 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:40:21.355 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:40:21.393 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T23:40:21.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:40:21.417 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:40:21.423 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:40:21.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T23:40:21.501 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T23:40:21.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:40:21.513 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:40:21.526 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:40:21.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T23:40:21.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.565 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-09-21T23:40:21.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.580 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-09-21T23:40:21.591 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-09-21T23:40:21.683 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:40:21.732 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:40:21.737 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:40:21.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:40:21.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T23:40:21.752 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T23:40:21.753 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi079 2024-09-21T23:40:21.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:40:21.797 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi118 2024-09-21T23:40:21.797 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:40:21.818 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914530/remote/smithi165 2024-09-21T23:40:21.818 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:40:21.839 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T23:40:21.839 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:40:21.842 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:40:21.859 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:40:21.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T23:40:21.897 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T23:40:21.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T23:40:21.950 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T23:40:21.950 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:40:21.953 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:40:21.955 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:40:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: 658458 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:40 /home/ubuntu/cephtest 2024-09-21T23:40:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:40 /home/ubuntu/cephtest 2024-09-21T23:40:21.961 INFO:teuthology.orchestra.run.smithi165.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:40 /home/ubuntu/cephtest 2024-09-21T23:40:21.963 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T23:40:21.983 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T23:40:22.052 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} duration: 800.2886834144592 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T23:40:22.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T23:40:22.128 INFO:teuthology.run:pass